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Abstract 



UNDERSTANDING THE LOW MATHEMATICS ACHIEVEMENT 
OF CHILEAN STUDENTS: 

A CROSS-NATIONAL ANALYSIS USING TIMSS DATA 
Dissertation by Marfa Jose Ramirez 

Advisor: Ina V. S. Mullis, Ph.D. 

The low performance of Chile in the TIMSS 1999 international study of 
mathematics and science achievement was a great disappointment. To investigate the 
likely causes for low performance in mathematics, this study 1) compared Chile to three 
countries and one large school system that had comparable economic conditions but 
superior mathematics performance, and 2) examined how important characteristics of the 
Chilean educational system could account for poor student achievement in mathematics. 
The results show that, compared to South Korea, Malaysia, the Slovak Republic, and 
Miami-Dade County Public Schools, Chilean 8 th graders had parents with fewer years of 
schooling and with fewer educational resources at home. At school, Chilean students 
were taught by teachers who felt less prepared to teach, and who covered fewer advanced 
mathematics content in class than teachers in other countries. Analyses using a series of 
hierarchical linear models show that, in Chile, school assets were unequally distributed 
across social classes. Schools in socially advantaged areas had more instructional 
resources and better prepared teachers; these teachers, in turn, emphasized more 
advanced mathematics content. Schools with their own mathematics curriculum and 



whose teachers covered more advanced content had significantly higher student 




achievement in mathematics. This relationship held true even after controlling 
statistically for the socio-economic level and type of administration (public/private). 
Regardless of school characteristics, students who a) expected to graduate from 
university, b) thought that doing mathematics was not so difficult, and c) thought that 
their academic performance did not depend on good luck or innate talent, attained 
significantly higher mathematics achievement. 
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CHAPTER 1 



INTRODUCTION 

Compared to other countries with similar economic conditions, the Chilean 
education system is lagging in terms of educational achievement. Data from the 1999 
Trends in International Mathematics and Science Study (TIMSS) show that other 
countries with similar economic resources manage to perform substantially better than 
Chile in both mathematics and science. This is the case in South Korea, Malaysia, and the 
Slovak Republic (Mullis, Martin, Gonzalez, et al., 2000). In other cases, United States 
districts serving high proportions of low-income students do significantly better than 
Chile, the Miami-Dade County of Public Schools in Florida being a good example 
(Mullis, Martin, Gonzalez, et al., 2001). 

These jurisdictions are like Chile in various ways. Chile, Malaysia, and the 
Slovak Republic are all upper middle-income economies according to the World Bank 
classification (World Bank, 1999). South Korea already reached the status of high- 
income economy; nevertheless, this country is not so far away from Chile in its economic 
development. While Miami-Dade County is part of United States - the strongest 
economy in the world - public schools in this county serve a significant proportion of 
low-income students. 

Despite these similarities in economic level, enormous differences exist in the 
academic performance of the students in these jurisdictions. While Chile obtained an 
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average score of 392 scale-points in the TIMSS 1999 mathematics test, South Korea 
scored 587, the Slovak Republic 534, Malaysia 519, and Miami-Dade 421. Considering 
that this test had an international average of 487 and a standard deviation of 100, these 
differences are substantial. 

The school effectiveness literature refers to several factors that may be helpful to 
understand why some schools attain higher performance levels than others. Some of the 
most frequently mentioned factors are: a) the broader social and economic context in 
which the schools operate, b) the curriculum, c) teacher quality, d) school resources, and 
e) students’ attitudes (Reynolds & Teddlie, 2000). These factors may be also helpful in 
understanding the achievement gap between Chile and the mentioned jurisdictions. 

While Chile shares similar economic indicators with South Korea, Malaysia, the 
Slovak Republic and Miami-Dade County, it differs from them in other important ways. 
For instance, as of 2003, only half of the adult population has finished secondary school 
in Chile (Ministerio de Planificacion y Cooperacion [MIDEPLAN], 2004). It is well 
known that parents’ education is one of the most powerful predictors of students’ 
academic achievement. Therefore, it is reasonable to hypothesize that the poor 
educational level of the students’ parents is limiting the ability of Chilean students to 
learn in school. It may be the case that other jurisdictions have a more educated adult 
population, and that these differences explain the achievement gap between Chile and 
other jurisdictions. 



2 




Among jurisdictions with similar global economic indicators, important social 
differences may still exist as a consequence of the distribution of the wealth. For 
example, the distribution of wealth is more uneven in Chile than in the Slovak Republic. 
Consequently, there may be a greater proportion of poor students in the Chilean schools 
than in the Slovakian ones. These differences in the students’ social background may 
account for their low academic performance. It is also likely that proportionally more 
students in Chile attend schools in rural areas than do students in either Miami-Dade or 
South Korea. Such demographic differences may also partially account for the lower 
achievement levels observed in Chile. 

In 1998, a limited mathematics curriculum was the basis for instruction in Chile 
(Comision Nacional para la Modemizacion de la Educacion, 1995; Cox, 1999). 1 The 
national curriculum was more of a framework for the schools to develop their own 
curricula than a curriculum itself. Consequently, it provided very little guidance for the 
mathematics teachers to prepare their classes. Moreover, some important content and 
skills were not part of the curriculum, and others were not emphasized enough. It may be 
the case that other jurisdictions had a more demanding mathematics curriculum, or a 
curriculum that provided more guidance to the teachers to prepare their classes. These 
differences in the official curriculum could be helpful in understanding the achievement 
gap between Chile and the other jurisdictions. 



1 The TIMSS 1999 data collection for Chile and the other southern hemisphere countries took place at the 
end of 1998. 
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Beyond the curricular intentions, the curriculum delivered in classes is the one 
that actually shapes the students’ opportunities to learn mathematics. There is 
accumulated evidence that the same official curriculum may result in different 
implemented curricula (Mullis, Martin, Gonzalez, et al., 2000, chap. 5). The implemented 
curriculum may vary in content coverage and cognitive skills emphasized in classes. 
These differences in the implemented curriculum could contribute to the differences in 
academic achievement between Chile and other jurisdictions. 

Because of the meager specifications of the Chilean curriculum, schools with their 
own curriculum or program of study, materials and activities may have been in a better 
position to produce quality learning. It may also be the case that the curriculum delivered 
in the Chilean classes provided fewer opportunities to learn the relevant mathematics 
content, compared to the curriculum delivered in other jurisdictions. It may also be the 
case that the Chilean schools offered fewer additional courses to address the needs of 
students with different knowledge and skills. These differences in the mathematics 
curriculum as it is implemented in the schools may be negatively affecting the quality of 
learning in Chile. 

In Chile, teachers are perceived as not having the basic knowledge and skills to 
bring their students to high academic standards. Their low social status may affect their 
salary, which may affect their motivation to work, which in turn may affect teaching 
quality and students’ learning rates. The poor academic performance of Chilean students 
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may be related to the low qualifications and preparation to teach of their teachers, relative 
to the teachers in other countries. 

The capacity of the schools to provide quality instruction can be affected by their 
resources. The TIMSS 1999 discusses important differences in school resources, both 
within and across the countries (Mullis, Martin, Gonzalez, et al., 2000, chap. 7). These 
differences may account for the achievement differences between Chile and the other 
jurisdictions. 

It could also be hypothesized that Chilean students differ from their peers in other 
countries in their attitudes toward mathematics. Chilean students may not like 
mathematics, or they may not appreciate its importance. The students may have a poorer 
self-concept of their mathematics knowledge and skills. They may also perceive that their 
mathematics performance depends more on factors out of their control, such as good luck 
or innate intelligence, and less on their own effort and motivation to do well. It is also 
possible that Chilean students have lower expectations for further education 
(e.g., finishing university) than the students in other jurisdictions. These differences in 
students’ attitudes may account for some of the dispersion in average mathematics 
achievement among the jurisdictions. 

While the achievement differences between Chile and other jurisdictions are 
large, the variations in students’ performance within the country also are substantial. 
According to the TIMSS 1999 report, the top performing Chilean students (i.e., those at 
national percentile 95) outperformed the bottom performing ones (i.e., those at national 
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percentile 5) by 280 score points (Mullis, Martin, Gonzalez, et al., 2000, p. 354). In 
Chile, the top performing students were able to apply basic mathematical knowledge in 
straightforward situations, while students with the lowest scores were not even able to do 
basic computations with whole numbers. 

There is evidence that the Chilean schools differ in important ways, and it is 
likely that these differences may account for some of the variation in students’ 
performance. First, the schools differ in the broad social context in which they operate. 
For instance, its is not the same to attend a school in the elite-paid system or a private- 
subsidized school as it is to attend a public school. Second, there are variations in the way 
the curriculum is implemented at the school and classroom level, in the provision of 
differentiated mathematics instruction, in teacher quality, and in school resources. 

There is variation in students’ achievement within the Chilean classes and 
schools. The students attending a school may differ widely in their attitudes toward 
mathematics. Some students may be highly motivated to do well while others may not. 
Some may believe that doing well in mathematics is a matter of luck, while others may 
rely more heavily on their own effort to do well. These differences in students’ 
characteristics may be helpful to understand why students who attend the same classes 
and schools vary widely in their mathematics performance. 
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Statement of the Problem 



This study aims to understand further why Chile has low mathematics 
achievement levels relative to other jurisdictions with similar economic conditions. It also 
aims to understand the achievement differences observed among and within the Chilean 
schools. 

This dissertation study used TIMSS 1999 data from Chile, and from four other 
countries and districts that have similar levels of economic development, or that serve 
high proportions of low-income students. The comparison jurisdictions were South 
Korea, Malaysia, the Slovak Republic, and Miami-Dade County Public Schools. The first 
stage of the study consisted of a comparative analysis between Chile and the four 
jurisdictions. The comparisons focused on five dimensions: a) schools’ community 
contexts, b) curriculum implementation, c) teacher quality, e) school resources, and 
f) students’ attitudes toward mathematics. Univariate and multivariate analyses of 
variance were the main statistical techniques used at this stage. 

The second stage of the study focused on the Chilean data only. Mathematics 
achievement was modeled using several indicators of the background dimensions above 
mentioned as predictors. The achievement outcome was modeled at two levels: the 
school level and the students-within-school level. Hierarchical linear models were the 
main statistical technique used. 
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Research Questions 



The research questions that guided the analyses were as follows: 

Schools’ community contexts: 

1. Are the schools’ community contexts poorer in Chile than in the comparison 
jurisdictions? Is there a positive relationship between schools’ community 
contexts and mathematics achievement in Chile? 

Two hypotheses guided the analyses: a) the schools’ community contexts are 
poorer in Chile than in the comparison jurisdictions, b) there is a positive 
relationship between schools’ community context and achievement in Chile. 

Curriculum implementation: 

2. Is the implemented mathematics curriculum poorer in Chile than in the 
comparison jurisdictions? Is there a positive relationship between the level of 
implementation of the mathematics curriculum and achievement in Chile? 

Two hypotheses guided the analyses: a) the implemented mathematics 
curriculum is poorer in Chile than in the comparison jurisdictions, b) there is a 
positive relationship between the level of implementation of the mathematics 
curriculum and achievement in Chile. 
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Teacher quality: 



3. Is teacher quality worse in Chile than in the comparison jurisdictions? Is there 
a positive relationship between teacher quality and mathematics achievement 
in Chile? 

Two hypotheses guided the analyses: a) teacher quality is worse in Chile than 
in the comparison jurisdictions, b) there is a positive relationship between 
teacher quality and mathematics achievement in Chile. 

School resources: 

4. Are the Chilean schools more poorly equipped than the schools in the 
comparison jurisdictions? Is there a negative relationship between resource 
limitations and mathematics achievement in Chile? 

Two hypotheses guided the analyses: a) the Chilean schools are more poorly 
equipped than the schools in the comparison jurisdictions, b) there is a 
negative relationship between resources limitations and mathematics 
achievement in Chile. 

Students’ attitudes toward mathematics: 

5. Are students’ attitudes toward mathematics more negative in Chile than in the 
comparison jurisdictions? Is there a positive relationship between students’ 
attitudes toward mathematics and achievement in Chile? 

Two hypotheses guided the analyses: a) Chilean students’ have more negative 
attitudes toward mathematics than the students in the comparison 
jurisdictions, and b) there is a positive relationship between students’ attitudes 
and achievement in Chile. 



9 




Importance of the Study 



The main purpose of this dissertation study is to contribute to the debate in Chile 
about how to improve education in general, and mathematics education in particular. In 
Chile, many proposals have been made to improve the quality of education. 
Unfortunately, many of these proposals are based on solely theoretical or ideological 
grounds, and have little to no empirical support. 

In Chile, studies using nationally representative samples are scarce, and are 
almost always limited to the data provided by the national assessment system, SIMCE 
(. Sistema de Medicion de Calidad de la Educacion). This dissertation study involves a 
comprehensive analysis using the data from an international study. TIMSS collected data 
on mathematics achievement, and on hundreds of contextual variables pertinent to 
understanding the academic performance of a country. 

Apart from the national report published by the Ministry of Education, there are 
few publications using the TIMSS dataset in Chile. These publications are usually 
focused on very specific aspects of the school system, or are restricted to descriptive 
analyses of the data already presented in the TIMSS reports. It is likely that this 
dissertation study is the first attempt to analyze and understand many of these 
background variables. Comparative analyses with other countries/systems are scarce in 
Chile. This dissertation is probably the first using a comparative approach to further shed 
light on the reasons underlying the poor mathematics performance of Chilean students. 
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CHAPTER 2 



LITERATURE REVIEW 

This chapter is divided into three main sections. The first section, the schooling 
system in Chile, describes the major characteristics of the educational system that is the 
focus of this dissertation study. The second section, the schooling system in the 
comparison jurisdictions, describes the main characteristics of the educational systems 
that were used as comparison cases in this study. The third section, factors related to 
mathematics achievement, reviews what the specialized literature says about the 
relationship between some relevant background variables and mathematics achievement. 

The Schooling System in Chile 

This section opens with an historical overview of Chile, then focuses on the main 
characteristics of its educational system: enrollment, school cycles, school governance 
and finances. The antecedents and main characteristics of the reform efforts of the last 
decades are provided. The official mathematics curriculum for the 8 th grade is described. 
Since this dissertation used data collected in Chile in 1998, the descriptions offered here 
aim to portray the state of the Chilean educational system at that time, even though some 
reforms have since been introduced. This section ends with an overview of the main 
findings provided by different international studies in which Chile has participated. 
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Chile is a long and narrow country that lies in the western coast of the South 
American Cone. Native tribes of gatherers, hunters, and fishers were scattered in its 
territory when the Spanish conquerors arrived in the 16 th century. Centuries of racial 
mixture with Spanish and other European immigrants have produced a population highly 
homogeneous in its ethnic background and language (Spanish). 

Chile gained its independence from Spain in 1810. It had a long tradition of 
democratically elected governments, which was broken with Pinochet’s coup in 1973. A 
right wing dictatorship was in power until 1990. A new democratically elected president 
took office that year, and a center-left coalition has been in the government since. During 
the last two decades, Chile has witnessed sustained economic growth; this is an upper 
middle-income economy according to the World Bank classification. At the end of the 
1990s, Chile had a population of almost 15 million, 84% of which were urban (World 
Bank, 1999). 

The Chilean educational system expanded substantially during the last decades. 
Enrollment in preschool education reached 21% in 1990, and rose to 30% in 1998 
(MIDEPLAN, 2004). Enrollment in primary schools reached 93% in 1970, 95% in 1982, 
and 98% in 1992. Enrollment in secondary schools reached 50% in 1970, 65% in 1982, 
and 80% in 1992 (Gonzalez, 1999, p. 323). The average number of school years of the 
economically active population doubled in 20 years, passing from 4.3 years in 1970 to 
8.6 years in 1990 (Rodriguez, 1995). As of 2003, half of the adult population had finished 
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secondary school (MIDEPLAN, 2004). Literacy rates increased from 70% in 1940, to 
90% in 1970, and 95% in 1990 (Rodriguez, 1995). 

School cycles and enrollment. In 1998, the Chilean schooling system served a 
population of slightly more than three million students, in approximately 10,000 schools 
(Ministerio de Educacion [MINEDUC], 2002a). The school cycle is divided into three 
levels: preschool, primary (grades 1-8) and secondary education (grades 9-12 2 ). 
Secondary education has two tracks: academic and professional-technical. Primary 
education has been compulsory since 1920; secondary education has been so since 2004. 

The majority of Chilean schools (56%) are located in an urban setting; these 
schools serve almost 90% of the student population. Rural schools are isolated and small; 
many of them have multi-grade classes, and are managed by three or few teachers. Rural 
areas frequently have higher levels of poverty, making it more difficult for rural schools 
to reach the desired educational goals. 

Governance and finances. Chile has a centralized educational system, with the 
majority of policy-related decisions made by the Ministry of Education. The Ministry of 
Education is the main authority regarding policies, curriculum, and selection of 
textbooks. It also negotiates the minimum salaries to be earned by school teachers 
throughout the country with the teachers’ union. The Ministry operates through a network 
of provincial and local (municipal) offices of education ( Departamentos Provinciales de 



2 Grades 9-12 correspond to “I-IV ano de ensefianza media” in Chile. 
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Education and Direction de Administration de Education Municipal), the latter being in 
more direct contact with the schools. 

A market-oriented philosophy underlay the financing and administration of the 
Chilean schools. In 1981, the military regime implemented a subsidized system, with the 
state paying a fixed amount of money to the schools (indirect vouchers) on the basis of 
enrollment. The main objective of this policy was to increase the efficiency of the 
system; that is, to produce the same school quality at lower costs. To what extent this 
goal was met is not clear due to the lack of good measures of school quality. 3 

During the 1980s, the number of private schools operating in the country grew 
consistently, and the proportion of students enrolled in the private system increased from 
around 10% to 40%. Most of these students (around 30%) enrolled in the new private- 
subsidized schools, which receive the same funding as public (municipal) schools, but 
have a private administration. During the 1990s, private-subsidized schools were allowed 
to charge fees to the parents; the schools that do so form the private-subsidized co- 
financing system. Public schools are funded by the state only. “By law and in practice, 
municipal education must guarantee compliance with free, compulsory education as 
established by the national constitution” (OECD, 2004, p. 68). A third group of schools is 
the elite-paid system. Elite-paid schools have historically served the elite 10% of the 
population, charge fees well above the voucher value, and do not receive public funding. 



3 The Chilean national assessment system, SIMCE, was created in 1988 and it only provided relative 
measures of academic achievement (norm-referenced tests) until the late 1990s. 
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Currently the educational system is highly segregated by social class: public 
schools serve the bottom half of the socio-economic distribution, the elite-paid system the 
top percentages, and the private-subsidized system the rest. Not surprisingly, public 
schools have consistently attained the poorest results in the national assessment system, 
followed by the private-subsidized schools. The best performing schools have been, by 
far, the elite-paid. There has been concern about the negative consequences of the 
subsidized system. Mella (2003) argued that it has increased the achievement gap 
between public and private schools, while exacerbating the social class differences 
among students from different schools. 

The privatizations do not seem to have produced higher academic achievement in 
the schools. There is evidence that public and private schools serving students with the 
same socio-economic backgrounds attain similar academic performance (McEwan & 
Camoy, 2000; Mizala & Romaguera, 2000; Ramirez, 2003). 

Regarding school finances, as of 1998, the voucher varied around US$40 per 
month per student, depending upon factors such as school cycle (i.e., primary, secondary) 
and context (i.e., rurality, poverty) (Gonzalez, 1999). Public annual expenditures per 
student reached US$1,767 in lower secondary (grades 7-8) (OECD/UIS, 2003). 

The percentage of the gross national product (GNP) that a country invests in 
education is an indicator of the country’s effort to provide quality instruction. In Chile, 
public expenditures in education have shown a positive trend during the 1990s, moving 
from a tiny 2.7% of the GNP in 1991 to a more reasonable 3.6% in 1997 (UNESCO, 
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1999, p. II 500). Nevertheless, Chile still had the lowest share of its gross national 



product invested in education when compared to the four jurisdictions. Table 2.1 presents 
a selection of social and economic indicators for Chile and the comparison jurisdictions. 



Data from the national assessment system (SIMCE) showed that, in the year 2000, 
the mathematics performance of 8 th grade students was very similar across the territory. 
All 13 regions but one had an average performance within one-tenth of a standard 
deviation from the national average (i.e., 250 ± 5 score points). The region with the 
highest poverty indices and with the highest proportion of indigenous population in the 
country (IX Region) had the lowest average performance, M = 237 (MINEDUC, n.d.-b, 
p. 22). Boys performed better than girls in the TIMSS 1999 mathematics test, but the 
difference was not significant (Mullis, Martin, Gonzalez, et al., 2000, chap. 1). 

The Chilean schools vary widely in their academic performance and in the social 
composition of the student body. This is in part the consequence of the economic 
segregation existing in the school system. Within the same schools, however, students are 
homogeneous in their social background. Despite sharing the same social class, 
classmates have very different levels of mathematics knowledge and skills. “While some 
children are ready to advance in their program of study, others are still trying to grasp the 
already-taught subject matters” (Ramirez, 2003, p. 20). This evidence suggests that social 
background conditions the students’ academic performance, but it is far from determining 
it. 
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Table 2.1 



Selected Social and Educational Indicators for Chile and the Comparison Jurisdictions 



Jurisdiction 


GNP 

per 

capita 1 


GNP PPP 
per 

capita 2 


Gini 

Index 3 


Human 

develop- 

ment 

index 4 


Public 

Expenditure 

on 

education 
as % of GNP 5 


Yearly 

expend- 

itures 

per 

student 6 


Gross 
enrolment 
ratio in 
primary 
education 7 


Net 

enrolment 
ratio in 
primary 
education 8 


Adult 

literacy 

rate 9 


Chile 


4, 820 


12,240 


57 . 1 


0.835 


3.6% 


1, 767 


103% 


88% 


95.7 


South Korea 


10, 550 


13,430 


31.6 


0.878 


3 . 7% 


3,208 


98% 


97% 


95.9 


Malaysia 


4, 530 


7, 730 


49.2 


0.789 


4.9% 


1, 813 


100% 


99% 


88 . 7 


Slovak Republic 


3,680 


7, 860 


25.8 


0.842 


5.0% 


1,811 


103% 


89% 


99 . 7 


Miami-Dade 


- 


- 


- 


- 


- 


6,613 


- 


- 


- 


United States 


29,080 


29,080 


40 . 8 


0.935 


5.4% 


- 


100% 


94% 


- 



1 GNP per capita = Gross National Product per capita, in US dollars. Source: World Bank (1 999) World development indicators, p. 12-14. 

2 GNP PPP per capita = Gross National Product Purchasing Power Parity per capita. An international dollar has the same purchasing power over the 
GNP as a U.S. dollar in the United States. Source: World Bank (1999) World development indicators, p. 1 2-14. 

3 Gini Index: Measures the extent to which the distribution of income (or consumption) among individuals or households within a country deviates 
from a perfectly equal distribution. A value of 0 represents perfect equality, a value of 100 perfect inequality. Source: United Nations Development 
Program. 

4 Human development index. A composite index based on a long and healthy life, knowledge and a decent standard of living. All data from year 
2000 with the exception of the Slovak Republic, where data from 2002 is reported. Source: United Nations Development Program. 

5 Source: UNESCO Statistical Yearbook (1999), p. II (500-51 1 ). 

6 OECD/UNESCO UIS WEI, 2003. Data for Miami-Dade County Public Schools was estimated by the author based on Statistical Abstract 2002-03 
(Miami-Dade County Public Schools, 2003, p. 34). 

7 Gross Enrolment ratio = Number of pupils enrolled in the given level of education, regardless of age, expressed as a percentage of the population 
in the relevant official age-group. Primary education = ISCED 1 . Source: UNESCO Institute for Statistics. 

8 Net Enrolment ratio = Number of pupils in the official age-group for a given level of education enrolled in that level expressed as a percentage of 
the total population in that age-group. Primary education = ISCED 1 . Source: UNESCO Institute for Statistics. 

9 Adult literacy rate. 2002 census data for Chile, Malaysia, and the Slovak Republic; data for South Korea was estimated. Source: United Nations 
Development Program. 

- Data not available. IV 




Educational Reforms During the 1990s 



In Chile, the return of democracy in 1990 brought with it a greater appreciation of 
the role education could play in both the life of Chileans and the development of the 
country. Universal enrollment was accomplished for the most part in primary education, 
and the vast majority of the population was finishing secondary school. However, the 
school system was perceived as ineffective, inefficient, inequitable, and unable to meet 
the requirements of a society that was looking forward to overcoming poverty and join a 
globalized economy. 

The newly elected president appointed a special commission to produce a 
comprehensive report of the state of education. This report, known as the Informe 
Brunner - after its primary author - provided a diagnosis of the educational situation, 
identified goals for the sector, and suggested strategies to reach them (Comision Nacional 
para la Modernizacion de la Educacion, 1995). An appalling picture of the condition of 
education was depicted; some alarming defects included: 

Students with poor knowledge and skills in language arts (Spanish), 
mathematics, and science 

Lack of specificity of the curricular frameworks and programs of study: 
objectives were ambiguous and performance standards nonexistent 

Poor pedagogical skills of the teachers; overemphasis of lecturing and lack of 
active work by students 

Poor school facilities 
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Inefficiencies in the administration of the system 



Low teachers’ salaries 
Low investment in education 

The Informe Brunner laid out the foundations of a systemic intervention that 
would take priority in the political agenda of the 1990s. Over time, this intervention came 
to be known as the Chilean educational reform. Its goals were quality and equity in 
education. Quality was pursued by striving for high standards of academic performance; 
equity was pursued by providing the technical and material conditions for everyone to 
reach those standards. The intervention focused on the schools that received funds from 
the state (public or private-subsidized). Priority was given to the poorest schools in 
receiving technical and material support (Garcra-Huidobro, 1994). The main areas of 
intervention of this reform were (Garcra-Huidobro, 1999, 2000; MINEDUC, 1998a): 

New up-to-date curricula for grades 1-12 
Increased instructional time 
Decentralization of pedagogical decisions 

Introduction of information communication technology in all the schools 
New teacher training programs (pre- and in-service) 

Increase in teachers’ salaries 

Allocation of more resources for the education sector 
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The launching of the reform agenda was welcomed enthusiastically from the 
different social sectors. However, the new curriculum was criticized. In Chile, there has 
been a constant tension between two opposing visions. On the one side, the Ministry of 
Education has been attempting to provide a general framework of learning objectives and 
content to be taught, together with documents aimed to provide a scaffolding to reach the 
curricular objectives (e.g., programs of study, pedagogical guides, performance 
standards). On the other side, there are those who advocate for the schools’ right to 
choose and produce their own curriculum and programs of study (see Alliende, 2000). 

A decade after the launching of this major education intervention, important 
progress has been made. Between 1990 and 1999, public spending on education increased 
2.5 times (MINEDUC, 2000a). All the secondary schools and more than half of the 
primary schools were supplied with computer labs connected to the Internet (Hepp, 
1999). By 1998, around 50% of the subsidized schools (both public and private) were 
able to increase the number of instructional hours. By 2002, new curricula had been 
introduced in all grades and content areas (Cox, 1999). 

Yet despite the reforms, there is no evidence that students are learning more. 
Comparable data from the national assessment of SIMCE 1999 and 2002 showed that the 
4 th graders made no progress in their achievement levels in mathematics, reading, and 
science (MINEDUC, 2003a). The available data for the 8 th grade led to similar 
conclusions when comparing the assessments from 1997 and 2000 (MINEDUC, 2001). 
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The assessment results also showed that a considerable gap exist between the 
goals set by the new curricular frameworks and what the students know and can do. This 
gap has been clearly identified in all three grades tested by the national assessment - 
grades 4, 8, and 10 (MINEDUC, 2003a, 2001, 2000b, 2002b). For instance, SIMCE 2000 
showed that only 45% of the 8 th graders could correctly answer a one-step, 
straightforward item that required the use of fractions and operations (MINEDUC, 2001, 
p. 26). Considering that fractions should have been introduced in the 3 rd grade 
(MINEDUC, 1980, 1998b) it is disappointing that five years later more than half of the 
students were still unable to use them in the simplest of applied contexts. 

Two main weaknesses are perceived to threaten reform in Chile. The first relates 
to the low qualification levels of the teachers. Teachers are perceived as not having the 
basic knowledge and skills to respond to the demands made by the reform project. There 
are widespread doubts about their capacity to bring the students to high achievement 
levels. Likewise, pedagogy is among the university programs that require the lowest 
scores in the entrance exams. 

The second weakness refers to the implementation of the new policies. The 
feeling is that the new curriculum is not reaching the classrooms. Teachers keep teaching 
basic mathematics content, put too much emphasis on the memorization of algorithms, 
and do not stimulate the development of higher order skills. Lecture teaching style 
dominates instruction, and students have few opportunities for active participation in 
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classes. Some efforts have been made to document the gap existing between the new 
reformed curriculum and its classroom implementation (MINEDUC, n.d.-a). 

The Official Mathematics Curriculum in Chile 

Curriculum reform has been at the core of the educational policies in Chile since 
1990s. A new curricular framework was published in 1996 and new programs of study 
for the 8 th graders were introduced in 2002. 4 However, in 1998, the year data was 
collected for this dissertation study, the curriculum in use at the 8 th grade was Planes y 
Programas de la Educacion General Basica (MINEDUC, 1980). This pre-reform 
curriculum was introduced during Pinochet’s right wing dictatorship, in 1980. It listed the 
minimum objectives to be reached by all the students. The mathematics objectives for the 
8 th grade were, de facto , brought to their minimal expression and were written in just 
three pages. 

During the authoritative regime, a sketchy policy instrument like this was thought 
“to free” the teachers and schools from the state structures, hence allowing them to make 
their own decisions regarding interpretation and implementation of the curriculum. These 
ideas were more clearly expressed in a law passed in 1990, at the very end of the military 
government. The Constitutional Law of Education (Ley Organica Constitucional de 
Ensenanza, LOCE ) was largely designed as a tool aimed to prolong the policies initiated 



4 See Cox (1999) and Valverde (2004) for an overview of curricular reforms in Chile. 
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during the 1980s. The law stated that the Ministry of Education is responsible for 
providing frameworks of fundamental objectives and minimum content. 5 

Based on these frameworks, schools have to develop their own curriculum or 
programs of study. Only the schools that did not do so could adopt the programs of study 
offered by the Ministry of Education. As of 1998, very few schools had developed their 
own curriculum, and most of these schools were elite-paid or private-subsidized. Most of 
the schools (especially the public ones) never had the necessary technical and material 
resources to do so. 

The curricular policies of the 1980s have been the target of important criticism. 
Espmola, and Gajardo and Andraca (as cited in Cox, 1999) claimed that the increased 
flexibility and decentralization of the curriculum ended up weakening its implementation. 
In practice, this policy translated into less instructional time and less content coverage for 
the schools that worked in impoverished contexts. 



The previous paragraphs described the political context and ideology underlying 
the curricular policy in Chile. In the following paragraphs, the focus turns to the 
pedagogical dimension of the curriculum and to the specificities of the mathematics 
curriculum used by the 8 th grade students. 



5 The Constitutional Law of Education ( Ley Organica Constitutional de Ensenanza [LOCE], 1990) defines 
broad educational goals and minimum skills to be reached by all the students. 
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The pre-reform curriculum listed the mathematics objectives to be reached by the 
Chilean students. The structure was simple: one general objective was followed by 
several specific objectives. For example, the students had to “know and apply the 
proportionality concept'’ (general objective), which translated into seven specific 
objectives, such as “compare two quantities and express them as a ratio,” and “recognize 
a proportion” (MINEDUC, 1980, p. 71). 

The Chilean curriculum shared many content areas and skills with the curriculum 
from other jurisdictions; however, it covered substantially less content and fewer skills. 
The TIMSS 1999 international report states that only 61% of the content in the TIMSS 
mathematics test was intended to be taught to Chilean students up through grade 8. This 
contrasted heavily with the situation in South Korea, Malaysia, and Miami-Dade County 
Public Schools, where 80% or more of the content was intended to be taught (Mullis, 
Martin, Gonzalez, et al., 2000, p. 172; Mullis, Martin, Gonzalez, et al., 2001, p. 82). 6 

The mathematics content and skills tested in TIMSS represent what the 
participating countries considered important to be learned at school up through grade 8. 
According to the test-curriculum matching analysis performed in TIMSS 1999, the 
Chilean mathematics curriculum had the lowest overlap with the test: only 58% of the 
test problems were considered appropriate for the 8 th grade students. In contrast, 75% of 
the problems were considered appropriate by the curriculum in South Korea, 83% by the 



6 Data were not available for the Slovak Republic. 
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curriculum in Malaysia, and 100% by the curriculum in the Slovak Republic and the 
United States (Mullis, Martin, Gonzalez, et al., 2000, appendix C). Table 2.2 summarizes 
some characteristics of the official mathematics curriculum in Chile and the comparison 
jurisdictions. 



Regarding emphasis on approaches and processes, the Chilean curriculum - as 
well as the curriculum of most of the TIMSS 1999 countries - put a major emphasis on 
mastering basic skills and on understanding mathematical concepts. However, the 
Chilean curriculum was among the very few that put little or no emphasis on solving non- 
routine problems and on assessing student learning (Mullis, Martin, Gonzalez, et al., 
2000, p. 163). The official curriculum also failed in promoting the development of 
important higher order thinking skills. For instance, mathematics reasoning was not part 
of the curricular intentions (MINEDUC, 1998b). See Table 2.2 for more detailed 
information on content and processes in the Chilean mathematics curriculum. 

The Chilean mathematics curriculum did not address differentiation for students 
with different abilities or interests, nor did it provide an explicit policy on the use of 
calculators. The Ministry of Education supported the implementation of the curriculum 
through mandated textbooks and notes. 
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Table 2.2 



Selected Characteristics of the Official Mathematics Curriculum in Chile and the Comparison Jurisdictions 





Emphases on approaches and processes 1 


Percent of 


Percent of 


Jurisdiction 


Basic 

skills 


Real-life 

applications 


Solving 

non-routine 

problems 


Assessing 

student 

learning 


TIMSS 

content 

appropriate 2 


TIMSS 

problems 

appropriate 3 


Chile 


• 


o 


o 


o 


61% 


58% 


South Korea 


• 


• 


© 


© 


80% 


75% 


Malaysia 


• 


• 


© 


© 


80% 


83% 


Slovak Republic 


- 


- 


- 


- 


- 


100% 


Miami-Dade County 
Public Schools 


• 


• 


• 


• 


93% 


100%* 



1 Source: TIMSS 1 999 International Mathematics Report, chap. 5; and TIMSS 1 999 Mathematics Benchmarking Report, chap. 5. 

2 Percent of TIMSS contents included in the official mathematics curricula up to the 8 th grade. Source: TIMSS 1 999 International 



Mathematics Report, chap. 5; and TIMSS 1999 Mathematics Benchmarking Report, chap. 5. 

3 Source: Test Curriculum Matching Analysis, TIMSS 1 999 International Mathematics Report, appendix C. 
* Data from the United States was imputed to Miami-Dade County Public Schools. 

- Data not available. 



£ Major emphasis q Moderate emphasis q Minor/ no emphasis 
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In Chile, there is a system-wide assessment in mathematics (SIMCE), with one 
grade (4, 8 or 10) tested each year. School level results are highly publicized and 
monetary awards are distributed (Mullis, Martin, Gonzalez, et al., 2000, chap. 5). 

Promotion to the next grade was supposed to be a joint decision between the 
teacher and school. In practice, students are automatically promoted to the next grade in 
primary education. Grade retention is practiced in extreme cases only - less than 3% of 
students were retained in grades 1-8 (MINEDUC, 2002a, pp. 172-173). Grade 12 students 
take an external exam to apply to the university. Riveros (2004) claimed that over 90% 
the students taking this exam had good or very good marks in their school records. This 
situation raised concern about the pervasive grade inflation existing in the country. It also 
corroborated the low expectations teachers have for their students, especially toward 
students come from socially deprived areas (Arancibia, 1994). 

Chile in the International Arena 

During the 1990s, Chile participated in three international studies in education: 
UNESCO’s Latin American Laboratory of Assessment of the Quality of Education, 
OECD’s Programme for International Student Assessment, and IEA’s Trends in 
International Mathematics and Science Study. By participating in these studies, Chile 
aimed to know the academic performance of the country relative to other participating 
countries. Chile also aimed to learn from the experiences of other countries, and to learn 
about the factors that may be affecting the quality of education. 
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A brief description of UNESCO’s and OECD’s studies is offered below, together 
with a summary of their most important findings for Chile. Because IEA’s TIMSS is the 
main data source for this dissertation study, its goals, design and results are reviewed in 
more detail. 

UNESCO’s Latin American Laboratory for 
Assessment of the Quality of Education 

' 7 

In 1997, UNESCO sponsored the Latin American Laboratory for Assessment of 
the Quality of Education ( Laboratorio Latinoamericano de Evaluation de la Calidad de 
la Education, LLECE), a study of mathematics and reading skills among 3 ld and 4 th 
graders in 13 countries of the continent (UNESCO, 2000a, 2000b). In both subjects and 
grades, Cuba outperformed all of the other countries, obtaining a median score almost 
two standard deviations above the international median. At the 4 th grade, Chile, 
Argentina, Brazil, Colombia, and Mexico averaged above the international median in 
mathematics, forming a cluster of countries behind Cuba. Unfortunately, the significance 
level of the differences among these countries was not reported, thus making it difficult 
to draw more meaningful conclusions. 

In mathematics, Chile had the highest score gain from the 3 ld to the 4 th grades. 
Students from private schools outperformed those in public schools, students from small 
cities did better than those in “mega-cities” (with more than one million inhabitants), and 
students from rural areas had lower scores than those in urban settings. This evidence 

7 UNESCO = United Nations Educational, Scientific and Cultural Organization. 
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suggests that Chile is in comparatively good shape regarding the quality of its educational 
system when looking at other Latin- American countries. 

OECD's Programme for International Student Assessment 

The Programme for International Student Assessment (PISA) is an international 
assessment project sponsored by the Organisation for Economical Co-operation and 
Development (OECD). Literacy in reading, mathematics, and science are measured in a 
3-year cycle; one of the three subjects is emphasized in each assessment. PISA measures 
how well 15-year-old students can use their skills and knowledge to answer questions 
based on real-life situations. Fifteen-year-old students are sampled regardless of the grade 
in which they are enrolled; however, only students enrolled in school had a chance to 
participate in the study. 

While PISA is targeted at OECD members - the richest countries in the world - 
some low- and middle-income countries administered PISA 2000, in a data collection 
effort that took place in 2002 (OECD, 2003). Chile participated in this effort with a 
nationally representative sample of students at the target age. In this sample, 61% of the 
15-year-old students came from the 10 th grade (MINEDUC, 2004, p. 224). 

Among the 41 countries participating in the assessment, Chile’s performance was 
relatively low in reading, mathematics, and science. Its average score was approximately 
one standard deviation below the OECD average. In both reading and mathematics, Chile 
ranked 36 th ; and in science Chile ranked 35 th (OECD, 2003). The Chilean results in 
mathematics indicated that, on average, the students only mastered the most basic 
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knowledge and skills evaluated by PISA. They could complete one step, straightforward 
problems that required the routine application of an algorithm to get the solution 
(MINEDUC, 2004, p. 103). 

Among developing nations, there is a substantial proportion of the youth 
population that does not attend secondary school. The higher this proportion, the more 
the country’s mean achievement reflects the knowledge and skills of an elite group of 
students rather than of the whole population. For this reason, enrollment rates are 
important to contextualize the results of international comparisons. Among the Latin 
American countries participating in PISA, Chile had the highest enrollment rates of the 
15-year-old population: 87%. While Argentina had three-fourths of its youth enrolled in 
the schools (76%), this proportion fell dramatically for Brazil and Mexico, where only 
one out of two youths were attending the schools (53% and 52%, respectively). 8 In 
reading, mathematics and science, the mean scores of Chile, Argentina and Mexico did 
not differ significantly. In contrast, Chile obtained significantly higher scores than Peru 
(in all three subjects) and Brazil (in mathematics and science) (OECD, 2003). 

Both South Korea and the United States had enrollment rates similar or higher 
than Chile (84% and 99% respectively). In mathematics, South Korea outperformed 
Chile by one and a half standard deviations (163 score points). The average Chilean 
students demonstrated knowledge and skills similar to the bottom performing students of 



s Author estimation based on Literacy Skills for the World of Tomorrow: Further Results from PISA 2000 
(OECD, 2003), p. 251, Table A3.1, columns 1 and 2. 
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South Korea. The United States outperformed Chile by one standard deviation (109 score 
points); and the top-performing students in Chile demonstrated knowledge and skills 
similar to the average students in the United States. 9 

lEA’s Trends in International Mathematics and Science Study 

Conducted by the International Association for the Evaluation of Educational 
Achievement (IEA), the Trends in International Mathematics and Science Study 
(TIMSS) 10 is a large-scale assessment program designed to study the performance of 
education systems around the globe. Monitoring trends is an important objective of the 
study, therefore the assessment is being repeated every four years. 

TIMSS 1999 collected information about students’ achievement in mathematics 
and science. The sampled students also answered a questionnaire providing background 
information. For example, the students were asked about their parents’ education and 
resources at home, their attitudes and beliefs toward mathematics, and their school 
experiences. The mathematics and science teachers of the sampled students provided 
background information about themselves, the contents covered in classes, and the 
pedagogical practices implemented while teaching. Principals from the sampled schools 
were asked about instructional provisions, school characteristics, and the student 
population served. Data about the intended mathematics curriculum, evaluation, and 



9 Malaysia and the Slovak Republic did not participate in PISA 2000. 

10 In the 1995 and 1999 evaluations, TIMSS stood for Third International Mathematics and Science Study. 



31 




system-level policies in general, were gathered through a curriculum questionnaire to 
provide a context to interpret achievement results. 

Thirty-eight countries participated in the study in TIMSS 1999. The 
Benchmarking Study was an additional component of TIMSS, whereby 27 jurisdictions 
in the United States (13 states and 14 school districts or consortia) took part to assess the 
standing of their students and education system from an international perspective (Mullis, 
Martin, Gonzalez, et al., 2001, p. 3). The target grade for most of the participating 
countries and for all the jurisdictions was the 8 th grade. 

Major Findings for Chile from TIMSS 1999 Mathematics. The Chilean 
mathematics performance in TIMSS 1999 was very poor. The average mathematics score 
was 392 score points, which was almost one standard deviation (100 score points) below 
the international average (M = 487). Chile ranked 35 th among the 38 countries 
participating in the study, only the Philippines, Morocco, and South Africa obtained 
lower average scores. Compared to the United States benchmarking participants, the 
Chilean results were the lowest (Mullis, Martin, Gonzalez, et al., 2000; Mullis, Martin, 
Gonzalez, et al., 2001). The average mathematics scores for the comparison jurisdictions 
used in this dissertation study were as follows: South Korea M = 587, the Slovak 
Republic M = 534, Malaysia M = 519, and Miami-Dade County Public Schools M = 421. 

As a national option, Chile also drew a nationally representative sample of 7 th 
grade students and tested them with the same mathematics test used at the 8 th grade. 
Ramirez (2004) reported a significant achievement gain between the two grades (33 score 



32 




points). This difference was also similar to that observed between Chile and Miami-Dade 
County Public Schools (392 versus 421 score points). The author estimated that Chilean 
students were three school years behind the countries performing near the international 
average in the TIMSS 1999. 

The TIMSS 1999 Chilean report (MINEDUC, 2003b) mentions background 
variables that may account for the lower average performance of the country. For 
instance, it suggested that Chilean students had fewer resources at home to support their 
education, that the mathematics curriculum covered fewer content areas than evaluated 
by the test, that teachers placed too much emphasis on teaching numbers (e.g., whole 
numbers, decimals), and that teachers did not feel confident teaching mathematics. 

Another relevant finding was that the Chilean performance was below 
expectations given its economic level. As shown in Figure 2.1, there was a positive 
relationship between mathematics achievement and the wealth of the countries. 
Nevertheless, countries with similar economic development presented substantial 
differences in their average achievement. For instance, South Korea scored almost 200 
score points higher than Chile, and the Slovak Republic and Malaysia scored more than 
100 score points higher than Chile. 

TIMSS 1999 showed that Chilean mathematics achievement was low relative to 
other countries. Moreover, Chile’s performance was dramatically low compared to the 
absolute standards set by the Chilean curriculum. An alarming finding was that only half 
of the students showed evidence of mastering basic computations (addition, subtraction, 
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rounding) with whole numbers. According to the Chilean curriculum, most of these skills 
should be mastered in the first cycle of basic education (grades 1-4). Only 15% of the 
students could apply mathematical concepts in straightforward situations. The low 
performance of Chilean students was pervasive across the five mathematics content areas 
evaluated by the test. 



Figure 2.1 

Mathematics Achievement by the Wealth of the Countries 




Note. The figure shows the 38 countries participating in TIMSS 1 999. Data for Miami-Dade County 
Public School was not displayed. Source: TIMSS 1 999 dataset. 

GNP PPP = Gross national product purchasing power parity as for 1 999. 
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The Schooling System in the Comparison Jurisdictions 



This section provides important context information to better understand the 
schooling systems in South Korea, Malaysia, the Slovak Republic, and Miami-Dade 
County Public Schools. For each jurisdiction, information about enrollment and school 
cycles, governance and financing, as well as on the official mathematics curriculum is 
provided. 



South Korea 

Located on the eastern coast of the Asian continent, Korea has a history of 
Buddhist and Confucian dynasties. The Chosun dynasty ruled the country beginning in 
1392, and the Korean alphabet was introduced during this reign. After five thousand 
years as a sovereign country, Korea became a Japanese colony in 1905. Independence 
came with the end of World War II, and with it the dramatic split of the country into 
North and South Korea (Gwang-Chol Chang, 1995; Se-ho Shin, 1995). 

The Korean educational system has a long tradition of formal education. Se-ho 
Shin (1995) related the following: 

The teaching of Confucianism was dominant in traditional schools before the 
advent of modem style schools in the late nineteenth century. During the Koryo 
dynasty and the Chosun period, higher civil officials were selected through a state 
civil examination system which tested their proficiency in Chinese classics. The 
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system made the learning of Confucian classics an educational tradition of the 
elite class, (p. 515) 

Western scholars have argued that Confucianism is at the core of the educational 
success of Asian countries (Paik, 2001; Stevenson & Stigler, 1992). In fact, Confucius 
(551-479 bc) was mainly a teacher, hence teachers are one of the most respected 
authorities in these countries. 11 Confucianism promulgates the importance of continuous 
learning as a way of self-development, which in turn is necessary for social harmony. 
Children are taught that persistence, effort and hard work are necessary to succeed both 
in school and in life (“Confucius and Confucianism,” 1993). 

Despite this tradition of learning, at the time of independence in 1945, only one- 
fourth of the adult population was literate, and less than 40% of the school-age children 
attended school (Gwang-Chol Chang, 1995). The history of the two Koreas has followed 
divergent paths since then. The following paragraphs focus on South Korea only - the 
comparison jurisdiction used in this dissertation study. 

South Korea is a highly homogeneous country both ethnically and linguistically. 
It has its own unique language (Korean) (Se-ho Shin, 1995, p. 515). At the turn of the 
millennium, the total population reached 46 million (World Bank, 1999). During the 
twentieth century, South Korea transformed itself from a feudal society into a highly 
industrialized nation. By the end of the 1990s, over 83% of its population was urban. 

11 Korea and other Asian countries celebrates “Teacher’s day,” which is a sort of Confucius day (Chung 
Park, Koran Institute of Curriculum and Evaluation [KICE], personal communication, October 26, 2004). 
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According to the World Bank classification, South Korea is high-income economy 
(World Bank, 1999). It is, in fact, the wealthiest of the five jurisdictions used for 
comparison in this dissertation study. 

South Korea accomplished lofty educational goals in the decades following its 
independence. Enrollment in primary school was universal for school age children by 
1960. In 1991, the enrollment rate was 100% for primary school, 95% for middle school, 
88% for high school, and 38% for higher education (Se-ho Shin, 1995). The present day 
population is highly educated. One out of four students have a parent who graduated from 
a university (Mullis, Martin, Gonzalez, et al., 2000, p. 254). The adult literacy rate is 96% 
and 89% of students attend preschool (World Bank, 1999). 

The Schooling System in South Korea 

School cycles and enrollment. At the time of the TIMSS 1999 data collection, the 
Korean educational system served a population of around 9 million of students in 
approximately 10,000 schools. The school cycle was divided into preschool, primary 
school (grades 1-6), middle school (grades 7-9), and high school (grades 10-12). High 
schools were divided into academic and vocational tracks. At the end of the 9 th grade, 
Korean students have to take examinations to enter high school. Usually, students with 
higher scores follow the academic track and those with lower scores follow the 
vocational one (Chung Park, Korean Institute of Curriculum and Evaluation [KICE], 
personal communication, October 12, 2004). In the 12 th grade, students take another 
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round of examinations to enter higher education. There is fierce competition for entry 
into the more prestigious institutions. 

In South Korea, education is highly valued as a means of social mobility (Se-ho 
Shin, 1995). Parents have a strong commitment to their children’s education and make 
huge efforts to support them during the school years. Tutoring and private lessons are 
widely spread practices that help children prepare for the external exams. Wolf (2002) 
reported that up to 50% of the middle school students attended tutoring after their regular 
school hours. Many Korean families are said to spend up to half of their income on 
private teachers ! 

Governance and finances. South Korea has a centralized education system, in 
which the Ministry of Education is responsible for formulating policies, directing 
subordinate agencies in planning and policy implementation, as well as publishing and 
approving textbooks. In 1999, yearly expenditures per student reached US$3,208 in 
middle schools (grades 7-9) (OECD/UIS WEI, 2003). Expenditures in education reached 
3.7% of the gross national product (GNP) (UNESCO, 1999, p. II 503). See Table 2.1 on 
page 17 for detailed information. 

The Korean system is highly privatized, with 28% of the students attending 
private middle schools, and 62% attending private high schools (Se-ho Shin, 1995; Kim, 
1997). The Ministry of Education governs both the public and the private system. This 
situation ensures that the same rules of the game apply to both types of schools. For 
instance, both type of schools receive public funding, charge a fee to the parents, and use 
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a lottery system to select their students. The only difference is that public schools are 

administered by the government structures, while private schools are usually under the 

administration of religious congregations (Park & Park, in press; Chung Park, KICE, 

12 

personal communication, October 26, 2004). 

The Official Mathematics Curriculum in South Korea 

The Korean national curriculum is revised periodically - its 6 th version was 
introduced in 1995. In some 170 pages, this document detailed the contents to be taught 
to the 8 th graders and provided examples of the kind of problems students must solve to 
demonstrate mastery (Ministry of Education South Korea, 1994). The curriculum 
emphasized the mastering of basic skills, as well as the application of mathematics to 
real-life problems. 

The Korean curriculum covered 80% of the content included in the TIMSS 1999 
test, and 75% of the problems in the test were considered appropriate for the 8 th grade 
students (Mullis, Martin, Gonzalez, et al., 2000, chap. 5). The school curriculum was 
uniform in all of the schools, and principals were responsible for monitoring its 
implementation at the classroom level (Se-ho Shin, 1995). See Table 2.2 (on page 25) for 
a comparison of the Korean curriculum with the curriculum of the other jurisdictions 
examined in this dissertation study. 



12 These and other regulations governing all publicly funded schools are part of the “Equalization Policy” 
introduced in 1974 with the aim to reduce the achievement gap among the Korean schools (Park & Park, 
in press). 
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In South Korea, the same curriculum was used for all students, but teachers were 



required to adapt it to accommodate students with different abilities or interests. The 
curriculum recommended the restricted use of calculators. The implementation of the 
curriculum was supported through pre- service and in-service teacher education, 
textbooks, instructional or pedagogical guides, ministry notes or directives, and a system 
of school inspection or audit. A system-wide assessment in mathematics was in place at 
grades 4-8, 10 and 11 (Mullis, Martin, Gonzalez, et al., 2000, chap 5). Within each level 
of school education, the promotion of students from grade to grade was generally 
automatic (Kim, 1997). 



Malaysia 

Malaysia has a strategic location in Southeast Asia, which has favored its 
development as a trade center. Small city-states scattered along the coast flourished since 
the 11 th century, and gradually fell under the dominance of the city-state of Malacca 
during the 15 th century. At that time, Malacca was already known in Europe as a strategic 
port for the spice trade. Malacca fell under Portuguese dominance in the 16 th century. The 
peninsula was a Portuguese, Dutch, and British colony before getting its independence 
from Britain in 1957 (“Malaysia,” 1993). 

The British colonial rule transformed the country socially and economically. 
Transportation networks and industries prospered. Thousands of Chinese and Indians 
migrated to work in mining, manufacturing, and rubber estates. A diverse society was 
thus created. Nowadays Malaysia is a multiracial, multicultural, multireligious, and 



40 




multilingual country. The main ethnic groups are Malays (some 60% of the population), 
Chinese (30%), and Indians (10%) (Aziz & Maimunah, 1995; “Malaysia,” 1993). 

At the end of the 1990s, the Malaysian population approached 22 million (World 
Bank, 1999). Malaysia is very rural; however, the urban population has been increasing 
rapidly: while 42% of the population was urban in 1980, this percentage increased to 
55% in 1997. Slightly more than half of the urban population is Chinese. Confucianism is 
the dominant philosophy/moral in the Chinese communities (Aziz & Maimunah, 1995; 
“Malaysia,” 1993; World Bank, 1999). 

Currently, Malaysia is a federal constitutional monarchy with a nonpolitical head 
of state, who is elected from among nine state hereditary rulers (“Malaysia,” 1993). This 
country has witnessed a strong economic boom during the last decades. The World Bank 
classifies Malaysia as having an upper-middle income economy, which is one of the 
strongest in Southeast Asia (World Bank, 1999). 

Being an extremely diverse country, the creation of a national identity has been 
one of the primary challenges of the educational system (Aziz & Maimunah, 1995; 
“Malaysia,” 1993). This is a difficult task because it has proven to be elusive (“Forging a 
Nation,” 2004): 

Children of different backgrounds do not even mix in schools, thanks to the 

Chinese and Indian communities’ insistence that children should study in their 

native language. About 90% of Chinese children attend state-financed Mandarin- 
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medium schools, while some 70% of young Indians are taught in Tamil. That 
leaves very few non-Malays studying in the mainstream public schools, where 
Malay is the main language of instruction.... So from an early age, Malaysians 
become accustomed to racial segregation, (p. 44) 

The Schooling System in Malaysia 

Enrollment and school cycles. In 1998, the Malaysian school system served a 
population of four million students. The school cycle is divided into preschool, primary 
(grades 1-6), lower secondary (grades 7-9), and upper secondary (grades 10-11). The 
majority of the population (68%) attends preschool. There has been universal coverage in 
primary schools since 1988. Coverage in lower secondary schools approaches 90% (Aziz 
& Maimunah, 1995). 

Governance and Finances. The Ministry of Education is the main authority at the 
federal level, responsible for formulating plans, programs, and projects. At the state level, 
the state education departments are responsible for the implementation of the federal 
mandates (Aziz & Maimunah, 1995). 

As for 1997, Malaysia expended 4.9% of its gross national product (GNP) in its 
educational system. While this percentage was not particularly high compared to the 
other jurisdictions, it is notable that for almost 20 years (from 1975 to 1994) education 
received over 5% of the GNP, with a peak of 6.6% of the GNP in 1985 (UNESCO, 1999, 
p. II 504). As shown in Table 2.1 (on page 17), yearly expenditures per student reached 
US$1,813 in secondary schools (grades 7-11) (OECD/UIS WEI, 2003). 
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The Official Mathematics Curriculum in Malaysia 



The National Curriculum in Malaysia was introduced in 1990. A 60-page 
document contained the mathematics specifications for the 8 th grade. These were divided 
into 13 learning areas, such as “directed numbers,” “square roots, cubes and cube roots,” 
“algebraic expressions n,” “circles,” and “statistics.” Some four pages were devoted to 
the treatment of each area; the following points were outlined (Ministry of Education 
Malaysia, 2003): 

Learning objectives, defining what should be taught in mathematical terms 
(e.g., understand and use the concept of coordinates) 

Suggested teaching and learning activities. 

Learning outcomes, defining what the student should be able to do 
(e.g., identify the x-axis, y-axis and the origin on a Cartesian plane). 

Points to note and vocabulary 

The Malaysian curriculum places major emphasis on the mastering of basic skills, 
as well as on daily applications of mathematical concepts. The curriculum emphasized 
problem solving, communication, reasoning, making connections, and application of 
technology. The importance of affective objectives such as “good moral values” and 
“patriotism,” the development of positive attitudes towards mathematics, and the 
appreciation of the “importance and the beauty of mathematics” were also strengthened 

(p. 1-9). 
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The Malaysian curriculum covered 80% of the content included in the TIMSS 
1999 mathematics test, and 83% of its problems were considered appropriate for the 8 th 
grade students (see Table 2.2 on page 25). The same curriculum was used for all students, 
but teachers were required to adapt it to accommodate students with different abilities 
and/or interests. Calculators were described as learning aids, however, the curriculum did 
not contain any recommendations about their use. 

The implementation of the curriculum is supported through pre- service and in- 
service teacher education, textbooks, pedagogical guides. Ministry of Education notes or 
directives, a system of school inspection or audit. Malaysia has a system of external 
examinations which is also aimed to support the implementation of the curriculum 
(H. Yusof and A. Zakaria, Ministry of Education Malaysia, personal communication, 
November 2, 2004). There is a system of external examinations to decide entry to course 
tracks (grade 9); certification and end of secondary education (grade 11), and certification 
of pre-university and entry to university (grade 13). In all publicly funded schools, 
students are automatically promoted up to grade 9 (Aziz & Maimunah, 1995; Mullis, 
Martin, Gonzalez, et al., 2000, chap. 5). 

Slovak Republic 

Only recently established in 1993, the Slovak Republic is a very new country with 
a long history. The fall of the Soviet block came with the division of Czechoslovakia, and 
with the adoption of a market economy. The 1990s was a period of profound economic 
and social transformations. Liberalization of the economy came with pervasive inflation 



44 




and unemployment. Despite these economic troubles, and thanks to the development of 
industries and agriculture in the decades following World War n, the people enjoy a high 
standard of living relative to other Eastern European countries (“Czechoslovakia,” 1993; 
Vantuch, 1995). Nowadays, the Slovak Republic is an upper middle-income economy 
according to the World Bank classification (World Bank, 1999). 

The Slovak Republic is a parliamentary democracy with a one-chamber 
parliament and a president elected by the parliament. At the end of the 1990s, the total 
population was 5.4 million (World Bank, 1999). The largest city, Bratislava, has less than 
half a million inhabitants. For an industrialized country, there is not a particularly high 
level of urbanization. Many Slovakians live scattered in rural farmlands or in villages in 
the mountains. In 1997, only 60% of the Slovak population was urban 
(“Czechoslovakia,” 1993; Vantuch, 1995; World Bank, 1999). 

Similar to other nations that were part of the socialist block, the Slovak Republic 
has a strong educational tradition. Universal enrollment in primary education was reached 
during the 1960s, and enrollment in secondary education reached 94% in 1996 (Vantuch, 
1995; World Bank, 1999). The population is highly educated: one out of five students has 
a parent who graduated from university (Mullis, Martin, Gonzalez, et al., 2000, p. 254). 
State preschools are available for children three to six year olds, and 76% of the 
population attends preschool (World Bank, 1999). 

Slovak parents are very concerned about their children’s education, and they 
make every effort to ensure quality instruction. Wolf (2002) reported the widespread use 
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of tutoring and private lessons in the country: more than three-fourths of the 8 th graders 



were enrolled in some kind of extra- school-instruction in mathematics! 

At the time of the TIMSS 1999 data collection, the morale of Slovak teachers 
seemed to be very low. During the 1990s, teacher salaries devaluated nine times in real 
terms (Vantuch, 1995). Only 12% of the students were taught by teachers who thought 
that society appreciated their work (TIMSS 1999 international database). 13 

The Schooling System in the Slovak Republic 

Enrollment and school cycles. At the time of TIMSS 1999 data collection, the 
school system served a population of almost one million students. The school cycle is 
divided into preschool, primary education (grades 1-9), and secondary education (grades 
10-13). Enrollment is almost universal, with 98% of the students entering secondary 
education. Secondary schools are divided in gymnasia (academic track), vocational and 
apprentice. Only 3% of the schools are private (Lukackova, 2002; Matusova, Berova, & 
Tothova, 1997). 



13 The percentage of students taught by teachers who felt that society appreciated their work was as follow 
for the other countries included in this dissertation study: Chile 46%, South Korea 54%, and 
Malaysia 86%. Data for Miami-Dade County Public Schools were not available (TIMSS 1999 
international database). 
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Governance and finances. In the Slovak Republic, the Ministry of Education was 
responsible for educational policy and its implementation, legislation and financing. In 
1996, school governance was shifted to local state administration. Departments of 
education at the district level are responsible for primary schools (Lukackova, 2002). In 
1999, yearly expenditures per student reached US$1,811 in the lower secondary schools 
(grades 5-8) (OECD/UIS WEI, 2003). The Slovak Republic makes a relatively strong 
effort to educate its people: 5% of its gross national product (GNP) goes to the 
educational budget (UNESCO, 1999, p. 11-511). As can be seen in Table 2.1 (on page 
17), this was the highest percentage when compared to the other jurisdictions included in 
this dissertation study. 

The Official Mafhemafics Curriculum in fhe Slovak Republic 

In the Slovak Republic, the official mathematics curriculum specifies the contents 
and performance standards to be reached by students in grades 1-9 (Balint, 2001). The 
performance standards for mathematics are divided into four areas: arithmetic, algebra, 
geometry, and data/statistics. For each area and grade level, the curriculum specifies the 
content and skills students are expected to master, and provides examples of the type of 
problems students must be able to solve. The percentage of students that must show 
proficiency is indicated. According to the Slovak curriculum, all the problems in the 
TIMSS 1999 test were considered appropriate for the 8 th grade students (see Table 2.2 on 
page 25). 
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The Slovak mathematics curriculum was used for all students, but teachers were 
required to adapt it to accommodate students with different abilities or interests. The 
implementation of the curriculum was supported through pre- service and in-service 
teacher education, textbooks, ministry notes or directives, and a system of school 
inspection or audit. At the end of each school year students were evaluated in different 
content areas and skills. The Ministry of Education set up the criteria for assessing 
student performance. At grade 8, students took an exam to determine acceptance to 
secondary school; at grade 12 another exam ( maturita ) was taken for certification 
purposes and entry to university. The Slovak Republic did not have system-wide 
assessments in mathematics (Matusova, Berova, & Tothova, 1997; Mullis, Martin, 
Gonzalez, et al., 2000, chap. 5; Vantuch, 1995). Students were automatically promoted 
from grades 1 through 9 (J. Kuraj, Ministry of Education Slovak Republic, personal 
communication, October 25, 2004). 

Miami-Dade County Public Schools 

The Miami-Dade County Public Schools comprise a district serving a low middle 
to low socio-economic area of Florida, the most southeastern state in the United States. 
Miami-Dade includes the southern half of the city of Miami - including downtown and 
the keys. The county has an estimated population of two million (Miami-Dade County 
Public Schools, 2004; U.S. Census Bureau, 2000). 

Miami is one of the fastest-growing metropolitan areas in the United States. Half 
of its inhabitants are foreign-born (U.S. Census Bureau, 2000). Immigration from Cuba 
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and, more recently, from other Caribbean and Latin American countries, have given the 
city a Hispanic identity. Many students enter the school system after migrating from their 
country of origin (M. Casares, personal communication, October 10, 2004). 

The Miami-Dade County Public Schools make up the fourth largest school district 
in the United States, with over 360,000 students enrolled in 325 schools. Public schools 
do not have a good reputation in the city of Miami. In more affluent communities, many 
parents prefer to enroll their children in private, fee-paying schools (M. Casares, personal 
communication, October 10, 2004). This fact is important in order to understand the 
social composition of public schools. In 1999, the percentage of students receiving free or 
reduced-price meals reached 56%, whereas the percentage classified as limited English 
proficient was 17%. The student population was 56% Hispanic, 31% African American, 
11% White, and 2% Asian and Native American (Thornton & Wongbundhit, 2002). 

In the 1994/95 school year, Miami-Dade County Public Schools launched a 
systemic reform intervention aimed to improve the quality of education. A new 
competency-based curriculum was implemented, low-track courses were eliminated, an 
algebra course was required for graduation, and professional development was offered at 
school sites. Thornton and Wongbundhit (2002) reported significant gains in students’ 
achievement as a result of this comprehensive intervention. 
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The Schooling System in Miami-Dade County Public Schools 



In the United States, education is a state and district responsibility. The district 
school board has direct jurisdiction over the schools and is responsible for matters related 
to preschool through secondary curriculum and instruction, educational planning and 
assessment, student services, and school operations. 

In Miami-Dade County, the school cycle is divided into preschool, primary 
education (grades 1-5), middle school (grades 6-8), and high school (grades 9-12). In the 
school year of 1998/99, Miami-Dade Public Schools yearly expenditures per student 
reached US$6,613 in grades 4-8. 14 

The Official Mathematics Curriculum in Miami-Dade County Public Schools 

There is no national curriculum in the United States. The states and districts 
usually develop their own curriculum. The Florida Sunshine State Standards for grades 6- 
8 in mathematics were introduced in 1996. These standards were expanded to include 
grade level expectations for the 8 th grade. In some 10 pages, these two documents 
specified the learning objectives to be reached by the 8 th graders in five areas: number 
sense, concepts and operations; measurement; geometry and spatial sense; algebraic 
thinking; and data analysis and probability. 



14 Author estimation based on Miami-Dade County Public Schools Statistical Abstract 2002-03, 
pp. 143-144. 
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The state also developed the Florida Curriculum Framework PreK-12 Sunshine 
State Standards and Instructional Practices, which is a guide for teachers to help students 
achieve the state standards. A hundred pages of this document are devoted to 
operationalize what it means to reach the standards in each grade. For each standard, it 
provides several benchmarks and sample performance descriptions showing the type of 
problems students should be able to solve at each level. The Florida’s Comprehensive 
Assessment Test was aligned with the state curriculum, and was administered at grades 5, 
8, and 10. Schools not performing well were provided with technical support (Mullis, 
Martin, Gonzalez, et al., 2001, chap 5). 

Beyond the state level policy instruments, the Miami-Dade County Public 
Schools developed the Competency-Based Curriculum, which was aligned with the state 
standards. This curriculum put a major emphasis on the mastering of basic skills, as well 
as on the mastering of more advanced processes, like real-life application of 
mathematical concepts, and solving non-routine problems. Regarding the fit of the 
curriculum and the TIMSS 1999 mathematics test, 93% of the contents in the test were 
covered by the curriculum and all its problems were considered appropriate for the 8 th 
grade students (see Table 2.2 on page 25). The curriculum recommended unrestricted use 
of calculators. Banks of activities, a website, and a CD-ROM with “best practices” were 
used to support the curriculum implementation (Mullis, Martin, Gonzalez, et al., 2001, 
chap 5). 
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Factors Related to Mathematics Achievement 

The purpose of this section is to better understand the factors that may affect 
academic achievement in general, and mathematics achievement specifically. Studies 
relating mathematics achievement to five major areas are reviewed. These areas are: 
a) the socio-economic level of the students’ families, b) the mathematics curriculum, 
c) teacher quality, d) school resources, and e) students’ attitudes toward mathematics. 

Socio-economic Level of the Students' Families 

The socio-economic level of the families has shown to be the strongest predictor 
of academic achievement. Its pervasive effect has been documented in hundreds of 
studies, the first and most influential being the Coleman Report (Coleman et al., 1966). 
Students with more educated parents, from more affluent families, and with more 
resources at home have systematically higher achievement levels than their peers from 
more disadvantaged homes. 

Using 8 th grade data from TIMSS 1995, Beaton and O’Dwyer (2002) studied the 
relationship between socio-economic status and mathematics achievement. They found 
systematic differences in the social status effect among the schools. For instance, in the 
United States, socio-economic status accounted for 75% of the mean achievement 
differences among the schools; in South Korea it accounted for 40%; and in the Slovak 
Republic it accounted for 24%. Within the classes, though, socio-economic status was not 
a strong predictor of the achievement differences among classmates; in fact, this variable 
only accounted for marginal amounts of outcome variance (less than 6% in South Korea, 
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the Slovak Republic and the United States). The authors concluded that “once the 
students are assigned to a classroom, having highly educated parents at home or having a 
computer in the home has little further to do with mathematical accomplishment” 
(P- 229). 

Using TIMSS 1999 data from Chile, Ramirez (in press) reported that socio- 
economic status accounted for most of the achievement spread among the classes from 
different schools. Yet within the classes, this variable was not a strong predictor of 
mathematics achievement. These findings were used as evidence of the social segregation 
occurring in the Chilean schools. The weak effect of socio-economic level within the 
classes was interpreted as the consequence of the homogeneous social background of 
classmates. Students seem to be already sorted in the schools based on the socio- 
economic level of their families. Since students within schools are homogeneous in their 
socio-economic level, it is likely that the effect of this variable was obscured when 
measured within the classes. 

In Chile, the educational system is highly stratified by social class: public schools 
serve the poorer families, private-subsidized schools serve the middle-class, and the elite- 
paid schools serve the richer families. Not surprisingly, the public system has consistently 
gotten lower scores in standardized achievement tests (MINEDUC, 2001). 

In this scenario, it is important to consider the effect of the socio-economic level 
of the communities served by the schools in order to better understand the achievement 
differences among the schools. Yet within the Chilean schools, students are 
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homogeneous in their socio-economic background. Hence, this factor cannot account for 
the large variations in achievement observed within the classes. 

In Chile, cultural differences are also reflected in the urban-rural dichotomy. 
Among schools with similar socio-economic levels, those from urban settings perform 
better than those from rural areas (Mizala & Romaguera, 2000; Ramirez, 2003). These 
differences may be a consequence of the schools having more access to technical support 
and material resources in the city. 

Mathematics Curriculum 

The curriculum is an important variable in understanding differences in academic 
performance because it shapes students’ opportunities to learn. The curriculum usually 
states content and skills to be learned, pedagogical approaches, and evaluation practices, 
as well as other aspects of the learning and instruction process. As shown in previous 
sections, there was wide variation in the curriculum documents when comparing Chile 
with the other jurisdictions included in this dissertation study. 

In Chile, the national curriculum was in fact a framework that provided very little 
guidance regarding what to teach and how to teach it. The schools were expected to 
develop their own curriculum or programs of study based on this framework. 
Unfortunately, the majority of the schools never had the necessary technical and material 
resources to do so. Hence, having a school curriculum became an important feature of the 
Chilean system. 
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Beyond the official (intended) curriculum, it is the curriculum implemented in 
classes that more directly shapes students’ experiences and opportunities to learn. IEA’s 
studies have developed a conceptual framework to understand the difference between the 
intended and implemented curriculum in cross-national contexts (see Robitaille, 1993). 
The discrepancies between the intended and the implemented curriculum arise from the 
necessary interpretation teachers make of the curricular intentions, and also from the 
classroom contexts in which they work. For instance, the implemented curriculum may 
vary depending upon teachers’ knowledge of mathematics, the previous knowledge of 
their students, and/or the availability of instructional resources. 

The gap between intended and implemented curriculum is large in Chile. While 
8 th grade students should be studying geometry and algebra in their mathematics classes, 
72% of Chilean students were taught by teachers who emphasized mainly numbers in 
their classes (e.g., whole numbers, fractions). Among the 38 countries participating in 
TIMSS 1999, Chile emphasized this content area the most. This emphasis came at the 
cost of not covering other important content included in the curriculum. For example, 
while simple algebraic expressions should have been studied beginning in grade 7 
according to the Chilean curriculum, only 68% of the students were taught this content by 
the end of grade 8. In contrast, more than 90% of the students in South Korea, Malaysia, 
the Slovak Republic, and Miami-Dade County Public Schools had an opportunity to learn 
this content (Mullis, Martin, Gonzalez, et al., 2000, chap. 5; Mullis, Martin, Gonzalez, et 
al., 2001, chap. 5). 
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Several studies have shown how opportunities to learn mathematics are related to 
socio-economic level and to academic achievement (Gau, 1997; Secada 1992; Cueto, 
Ramirez, Leon, & Pain, 2003). Students enrolled in schools serving more affluent 
communities are more likely to be taught greater content in their classes, more likely to 
have taken algebra courses, and more likely to be taught the content included in 
achievement tests, when compared to students attending school in less affluent 
communities. Secada (1992) provided further evidence of how these unequal 
opportunities to learn were related to race, ethnicity, and social class in the United States. 
Gau (1997) stated that less affluent students are in a doubly disadvantaged situation. The 
scarce educational support they get at home is further confounded by the fewer 
opportunities to learn they have at school. 

Because of the way the educational system is structured in Chile, there is serious 
concern about the unequal distribution of learning opportunities. Private schools 
(subsidized and elite-paid) satisfy the demands of the upper- and middle-income families. 
The children of these families benefit from additional educational support at home 
(e.g., more educated parents), and hence are in a better position to learn at school. On top 
of that, private schools can charge fees to the parents, thus increasing the school 
resources and their capacity to provide a quality education. On the contrary, public 
schools serve the children from the most socially disadvantaged backgrounds. Because 
public schools are not allowed to charge fees, they usually have less resources and 
resources of a poorer quality than their private counterparts. 
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Teacher Quality 



It is reasonable to assume that quality teaching requires professionals with formal 
training in both how to teach (i.e., education) and what to teach (e.g., mathematics). 
Fennema and Loef (1992) heavily emphasized the importance of content knowledge for 
effective teaching: 

Since one cannot teach what one does not know, teachers must have in-depth 
knowledge not only of the specific mathematics they teach, but also of the 
mathematics that their students are to learn in the future. Only with this intensive 
knowledge of mathematics can a teacher know how to structure her or his own 
mathematics teaching so that students continue to learn, (p. 147) 

According to the TIMSS 1999, while almost all the Chilean 8 th graders were 
taught by certified teachers, only 78% were taught by teachers having mathematics as 
their major area of study. This contrasted heavily with the situation in South Korea and 
the Slovak Republic, where 97% of the students were taught by teachers with a 
background in mathematics. Despite the high average mathematics achievement of 
Malaysia in the TIMSS 1999 test, only 72% of its students were taught by teachers with a 
background in mathematics (Mullis, Martin, Gonzalez, et al., 2000, p. 189). In Miami- 
Dade County Public Schools, only one-third of the students were taught by teachers who 
majored in mathematics (TIMSS 1999 Benchmarking database). 

Mayer, Mullens, Moore, and Ralph (2000) reviewed studies linking teachers’ 
academics skills (as measured by their entrance examination scores) and their students’ 
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scores in achievement tests. They reported that the academic skills of the teachers were 
positively correlated with their students’ achievement scores. This finding should be a 
concern in Chile considering that pedagogy is among the least selective university 
programs. 

In Chile, teacher training programs have been in crisis for years. They are among 
the less selective university programs and have a highly bureaucratic profile. The courses 
use outdated programs of study and lack connection with schools (Avalos, 1999). 
Because of these characteristics, the teaching profession does not have a high status. The 
poor training offered by these programs may explain why Chilean students taught by 
certified teachers who majored in mathematics attained achievement levels far below 
expectations in the TIMSS 1999 test. 

Beyond teachers’ formal studies, another relevant issue for quality teaching is 
how well prepared the teachers feel to teach different mathematic contents and skills. In 
this context, preparation to teach is a proxy of teachers’ knowledge of mathematics. It is 
expected that teachers who feel better prepared to teach would have the capability to 
produce higher achievement levels in their students than teachers who do not feel well 
prepared. This was in fact the achievement trend for most of the countries in TIMSS 
1999. 



TIMSS 1999 created an index of “teachers’ confidence in preparation to teach 
mathematics.” This index was based on teachers’ responses to 12 questions about how 
well prepared they felt to teach different aspects of mathematics included in the test. 
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Based on the teachers’ index scores, the students were classified in a high, medium, or 



low index level. In Chile, only 24% of the students were taught by teachers who felt very 
well prepared (high index level), compared to 87% in the Slovak Republic, 86% in 
Miami-Dade County Public Schools, 75% in Malaysia, and 48% in South Korea. Chilean 
students taught by teachers who felt very well prepared attained substantially higher 
achievement scores than students taught by teachers who did not feel as well prepared 
(Mullis, Martin, Gonzalez, et al., 2000, p. 192; Mullis, Martin, Gonzalez, et al., 2001, 

p. 206). 



Provasnik and Young (2003) raised the question of whether the most qualified 
teachers in the United States tended to move to schools with better performing students, 
leaving disadvantaged students with the worst teachers. To test this hypothesis, the 
authors studied the relationship between teacher quality and student background 
characteristics. Indicators of teacher quality were teacher education and self-confidence 
in teaching specific mathematical skills/competencies, among others. Student background 
characteristics included parents’ education and free/reduced-price lunch eligibility, 
among others. The authors reported that quality teachers were more likely to be found in 
desirable teaching jobs: in schools that offer higher salaries, that are well resourced, that 
do not have large number of students with special learning requirements, and that are not 
in remote areas. 

While these findings are hardly surprising, they have at least two major 
implications. First, they raise the question of equity in the distribution of opportunities to 
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learn. The more disadvantaged students are taught by the least qualified teachers, hence 
perpetuating a vicious cycle of poverty: the poorer the students are, the less likely they 
are to have a highly skilled teacher, and the less likely they are to get the skills that will 
allow them to overcome poverty. Second, these findings pose a technical problem for the 
study of teacher effect on academic achievement. Because teacher qualifications and 
students’ background variables are correlated, teacher effect is confounded with other 
background variables. 



School Resources 

The capacity of schools to provide quality instruction can be affected by their 
resources. The TIMSS 1999 report showed that there were important differences in 
school resources, both across and within the countries; and that these resources were 
correlated with mathematics performance. An “index of availability of school resources 
for mathematics instruction” was computed based on schools’ average responses to 
questions about shortages that affect mathematics instruction. Resources were classified 
into two categories: shortages that affect general capacity to provide instruction 
(e.g., school buildings and grounds), and shortages that affect mathematics instruction 
(e.g., calculators, computers). High index levels indicated that both shortages, on average, 
affected instructional capacity none or a little. Low levels indicated that both shortages 
affected instructional capacity some or a lot. Medium level encompassed all other 
combinations of responses (Mullis, Martin, Gonzalez, et al., 2000, p. 232; Mullis, Martin, 
Gonzalez, et al., 2001, p. 254). 
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Students in better-resourced schools outperformed their peers attending schools 
where the principal reported that instruction was negatively affected by shortages. In 
Chile, students in the high index level (22%) had an average mathematics score of 435, 
while those in the low index level (10%) scored 365. 

In South Korea, Malaysia, the Slovak Republic, and Miami-Dade County Public 
Schools, students in better-resourced schools also attained higher achievement than 
students in schools affected by more shortages. The results for South Korea, Malaysia, 
and the Slovak Republic differed from Chile in some interesting ways. These three 
countries had lower proportions of students attending well-resourced schools, compared 
to Chile. Also, students in the low index level did even better than Chilean students 
enrolled in well-resourced schools! 

A categorical index like the one used in TIMSS 1999 was appropriate for 
reporting differences in school resources among several countries. However, a continuous 
measure can probably better capture the real differences in school resources existing in 
Chile. Because of its increased variance, a continuous variable is also probably better 
than a categorical variable to model mathematics achievement. 

Students’ Attitudes Toward Mathematics 

In Chile, the debate about how to improve academic achievement usually points 
to structural factors such as teacher education programs, school resources, or curricula. 
However, little attention has been given to students’ perceptions, such as how much they 
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like mathematics, how important they think it is, how difficult they perceive it is to do 
well in mathematics, and what factors they believe affect their mathematics performance. 
By the same token, little is known about students’ aspirations for further education: Do 
Chilean students just want to finish secondary school or do they plan to enter university? 

Students’ opinions and beliefs regarding mathematics, how much they like and 
value it, and what they forecast for their own future education can all be understood as 
different facets of students’ attitudes toward mathematics. An attitude is an internal 
disposition to evaluate an object in positive or negative terms. It is accompanied by 
affective, cognitive, and behavioral responses (for a detailed discussion, see Aiken, 
2002). The study of attitudes toward mathematics is justified from at least three 
standpoints. First, the development of positive attitudes is a goal for many educational 
systems; they are seen as a requisite for students’ academic engagement and to boost 
learning. Second, attitudes are learned predispositions that reflect the school ethos and the 
wider social context in which mathematics instruction occurs. As such, attitudes can be 
influenced by policy interventions. Third, the literature has suggested that there is a 
positive relationship between favorable attitudes toward mathematics and academic 
competence. 

However, the study of attitudes in the school setting has several complications. 
Because of the different facets of the attitudinal construct, what is meant by attitudes 
toward mathematics varies from one study to the other. Moreover, it is common to find 
studies that do not use the term attitudes, but whose focus lay in one or more of its facets 
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(e.g., academic self-perception, locus of control). The unit of analysis is another 
complication in the study of attitudes. Relationships that may hold true at the student 
level may not be of the same nature or strength at the school or classroom level. 

From the TIMSS 1995 there is evidence that the majority of 8 th graders around the 
world liked mathematics, thought it was important for them to do well in this subject, 
thought it was not boring, and did not find it easy (Beaton et al., 1996; Kifer, 2002). In 
school effectiveness studies, expectations for further education had been reported as a 
strong predictor of school mean achievement across the countries (Martin, Mullis, 
Gregory, Hoyle, & Shen, 2000). In South Africa, students with higher self-concepts 
(i.e., who thought that it was not so difficult to do well in mathematics) and who valued 
the importance of mathematics, were more likely than their peers with low self-concepts 
to attain higher mathematics performance (Howie, 2002). 

Using data from the Chilean national assessment system, Ramirez (2003) found 
that 4 th graders with more positive attitudes toward their studies (as reported by their 
parents) attained substantially higher mathematics scores than students with more 
negative attitudes. By the same token, the more the parents expected their children to 
study in the university, the higher the child’s mathematics performance. Attitudes and 
educational aspirations were among the very few variables that made a substantial 
contribution to achievement, after controlling statistically for the effect of the socio- 
economic level of students’ families. 
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Cueto, Andrade, and Leon (2003) reported that primary students in Peru liked 
mathematics, and had a high concept of their self-efficacy in doing mathematics. 
However, these positive attitudes declined in secondary school, probably as a 
consequence of the increased difficulty of the mathematics content covered in classes. 
Students’ perception of the utility of mathematics remained high throughout the school 
years. 



In Israel, Nasser and Birenbaum (2004) studied the relationship between 
mathematics achievement and some learner-related variables, including self-efficacy, 
beliefs regarding knowledge, and attitudes toward mathematics. The authors conducted 
their study in two samples of 8 th graders: Arabs and Jews. They reported that in both 
groups the strongest predictor of mathematics achievement was students’ beliefs 
regarding their performance capabilities in mathematics (self-efficacy). The better the 
students evaluated themselves in doing mathematics, the higher their academic 
performance. 

The authors operationalized attitudes towards mathematics as encompassing how 
much the students liked mathematics, how difficult they found it, and how important they 
thought it was. The relationship between mathematics achievement and attitudes was not 
so clear. They found that attitudes had minor and insignificant effect on the mathematics 
achievement of Jewish children, while it had a modest but significant effect on the Arabs’ 
achievement. Arab students reporting more positive attitudes toward mathematics 
attained higher levels of performance than those reporting more negative attitudes. 



64 




Interestingly, while the Jewish students outperformed their Arab peers in mathematics, 
the latter reported more positive attitudes toward this subject. 

Directly related with the above findings, Shen and Pedulla (2000) found that more 
students from lower performing countries reported having more positive attitudes toward 
mathematics: they found it easier and perceived themselves as more capable of doing 
well in this subject. These authors interpreted their findings as the consequence of more 
demanding curricula and higher standards used in high achievement countries, which 
may lead the students to see mathematics as a harder content area, and themselves as less 
capable of doing mathematics. 

Benham (1995) reviewed studies relating students’ self-perceptions and academic 
achievement. When students believed that their academic performance was the 
consequence of their own actions (e.g., studying hard, perseverance, motivation) rather 
than the consequence of factors out of their control (e.g., good luck, innate ability), they 
had better academic performance. In the literature, the former are said to have an internal 
locus of control, while the latter have an external one. Stemler (2001) provided further 
evidence of the importance of the students developing an internal locus of control. From 
an analysis of 4 th grade TIMSS 1995 data from 14 countries, he concluded, “effective 
schools tend to have a student body who is able to see a connection between hard work, 
ability, and achievement, and are less likely to attribute achievement to factors such as 
luck” (p. 145). 
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CHAPTER 3 



METHODOLOGY 

This chapter documents the main technical aspects of this dissertation study. First, 
it presents the criteria used to select the comparison jurisdictions, and describes the 
TIMSS 1999 sample design and implementation. Second, it offers a description of the 
mathematics test and background questionnaires. Third, the focus turns to the exploratory 
analyses carried out to define which background variables to use, how to summarize 
them, and how to deal with missing data. The fourth section, data analysis, provides 
detailed information about the statistical techniques used in both the comparative and the 
national analyses of the Chilean data. 

Comparison Jurisdictions 

Selection Criteria 

The TIMSS 1999 (Gonzalez & Miles, 2001) and the Benchmarking Study 
(TIMSS 1999 Benchmarking Database) datasets served as the main sources of 
information for this study. From the 38 countries participating in TIMSS 1999, four were 
selected for this dissertation study. These were Chile, South Korea, Malaysia, and the 
Slovak Republic. TIMSS 1999 also included an ambitious benchmarking component by 
the United States involving 13 states and 14 school districts. One of these districts, the 
Miami-Dade County Public Schools, was also included in this study. While Chile was the 
focus of the study, the other four jurisdictions served as a basis for comparison for the 
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Chilean data. The comparison jurisdictions were selected based on the following criteria 
(Table 3.1): 

Their economic situation was similar to Chile’s 
They tested the same grade as Chile (grade 8) 

They sampled students with an equivalent average age to Chile (14 years) 

They outperformed Chile on TIMSS 1999 15 

As discussed in Chapter 2, each of these jurisdictions shared Chile’s background 
characteristics. They were middle-income economies, were in the high/upper-middle 
level of the human development index, and their education systems had universal 
coverage. Table 3.1 provides further information about Chile and the four comparison 
jurisdictions in regards to their participation in TIMSS 1999. 



15 It could have been interesting to include in this study a jurisdiction with a lower average performance 
than Chile. But of the three countries with average scores lower than Chile - Morocco, the Philippines, 
and South Africa - none met the criteria of sampling the same target grade, and same students’ average 
age than Chile. While Morocco and the Philippines tested the 7 th grade, in South Africa the 8 th grade 
students were one year older than the Chilean ones (15.5 year old). All the United States jurisdictions 
performed better than Chile in the TIMSS 1999 mathematics test. 
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Table 3.1 



Characteristics of Students’ Samples and Average Mathematics Achievement 



Jurisdiction 


TIMSS 1999 
math score 
(Std. error) 


Target 

grade 


Average 

age 


Chile 


392 


(4.4) 


8 


14.4 


South Korea 


587** (2.0) 


8 


14.4 


Malaysia 


519** (4.4) 


8 


14.4 


Slovak Republic 


534** (4.0) 


8 


14.3 


Miami-Dade County 
Public Schools 


421* 


(9.4) 


8 


14.3 



Pairwise comparison significantly higher than Chile at level * p < .05 and ** p < .01 . 
Sources: TIMSS 1 999 international Mathematics Report and TIMSS 1 999 Mathematics 
Benchmarking Report. 



Sampling Design and Implementation 

As is the case with all TIMSS assessments, to participate in TIMSS 1999 all of 
the education systems had to comply with stringent sampling policies. TIMSS 1999 
collected achievement and background information from nationally representative 
samples of students, as well as background information about their schools and teachers. 
The TIMSS 1999 international target population was defined as all students enrolled in 
the upper of the two adjacent grades that contained the largest proportion of 13-year-olds 
at the time of testing. This was the 8 th grade in most of the countries (Foy & Joncas, 
2000a, 2000b). 

TIMSS 1999 used a two-stage stratified sample design to sample from the target 
population. In the first stage, a sample of schools was selected using the probability- 
proportional-to-size (PPS) method. In the second stage, entire mathematics classes were 
randomly selected from the target grade in each sampled school. The sampling guidelines 



68 




required the countries to sample around 150 schools. The participants in the 



benchmarking study were required to sample 25 schools (Foy & Joncas, 2001; Fowler, 
Rizzo & Rust, 2001). Chile, Malaysia, and the Slovak Republic sampled one entire class 
per school, whereas South Korea and Miami-Dade sampled students from different 8 th 
grade classes. Table 3.2 presents information about sample implementation in the five 
jurisdictions included in this dissertation study. 



Table 3.2 



Sample Implementation 



Jurisdiction 


8 th grade 
student 
population 


Sampled 

schools 


Sampled 8 th 
grade 
students 


Percent of 
sampled 
students that 
participated 


Chile 


238, 894 


185 


5, 907 


96% 


South Korea 


635, 080 


150 


6,114 


100% 


Malaysia 


378, 762 


150 


5, 577 


99% 


Slovak Republic 


76, 790 


145 


3,497 


94% 


Miami-Dade County 
Public Schools 


24, 485 


25 


1,356 


91% 



In all the countries, data were collected at the end of the school year. For Chile - 
the only southern hemisphere jurisdiction included in this dissertation - this was 
November 1998. For South Korea, Malaysia, the Slovak Republic, and Miami-Dade - the 
northern hemisphere jurisdictions - data were collected between February and June 1999. 
Detailed information about the sampling procedures and other technical aspects of the 
assessment can be found in the TIMSS 1999 Technical Reports (Martin, Gregory & 
Stemler, 2000; and Martin, Gregory, O’Connor & Stemler, 2001). 
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As a consequence of the TIMSS 1999 sample design, each Chilean school was 
represented by one class, so achievement differences among the schools could not be 
distinguished from achievement differences among the classes from the same schools. 
For this reason, schools and classes are referred to as the school/class level or unit in this 
dissertation study. As described in a later section, this study modeled the average 
achievement of the Chilean schools/classes in an attempt to understand better their 
mathematics scores. 



Instruments 

The TIMSS 1999 Mathematics Test 

The TIMSS 1999 mathematics assessment framework contained two main 
dimensions: content and performance expectations (Mullis, Martin, Smith, et al., 2001). 
There were five content areas: fractions and number sense; measurement; algebra; 
geometry; and data representation, analysis and probability. The performance 
expectations included five cognitive skills: knowing, using routine procedures, using 
complex procedures, investigating and solving problems, communicating and reasoning. 

The TIMSS 1999 test was the product of a collaborative effort involving 
mathematics educators and specialists from all around the world. Every effort was made 
to ensure that the test represented the curricula of the participating countries and that the 
problems in the test exhibited no bias towards or against particular countries. The 
national research coordinators of the participating countries reviewed all of the problems. 
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The problems were extensively reviewed for accuracy, clarity, classification according to 
the framework, format, and number of score points (Garden & Smith, 2000; Mullis & 
Martin, 2000). 

The mathematics test consisted of 162 problems. Multiple-choice problems made 
up three-fourths of the items and were each worth one point. One-fourth of the problems 
were in a free-response format, requiring the students to write their answers and provide 
explanations. These problems were each worth one or two points. The problems were 
distributed across eight booklets according to a matrix sampling design with students 
responding to one booklet only. This matrix sampling design maximized the content and 
skills that could be evaluated by the assessment in the time available (Garden & Smith, 
2000 ). 



Problems were assigned to booklets so that each booklet contained some 
mathematics items. 16 This, in conjunction with item response theory (IRT) scaling, 
allowed students to be given a mathematics score regardless of the booklet answered. 
Because each student only answered a subset of the mathematics problems included in 
the test, there was more uncertainty associated with each student scores. Assuming that 
there is a random distribution of plausible scores associated with each student average 
score, TIMSS draws five random scores from that distribution. The TIMSS dataset 



16 Test booklets contained both mathematics and science problems. 
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provides five “plausible values” for each student. These scores are appropriate for 
making inferences about the student population (Yamamoto & Kulick, 2000). 

Trained scorers from each country scored the students’ responses using a two- 
digit rubric. The first digit indicated correctness level and the associated score points, and 
the second digit was a diagnostic score to track types of approaches, common errors, etc. 
The average inter-rater reliability was 99% for the first digit, and 96% for the first and 
second digit combined (Garden & Smith, 2000; Mullis, Martin, Gonzalez, et al., 2000, 
appendix A). 

The reliability of the test was estimated using the internal consistency approach. 
The median Kuder-Richardson Formula 20 (KR20) for all the countries was p KR20 = 

0.89. For the jurisdictions in this study, it was as follows: Chile p KR20 = 0.83, South 
Korea p KR20 = 0.91, Malaysia p KR20 = 0.90, the Slovak Republic p KR20 = 0.89, and 
Miami-Dade County of Public Schools p KR20 = 0.86 (Mullis, Martin, Gonzalez, et al., 
2000, appendix A; Mullis, Martin, Gonzalez, et al., 2001, appendix A). 

The TIMSS 1999 Background Questionnaires 

The purpose of the TIMSS questionnaires was to provide a context to help in 
better understanding the students’ mathematics performance. The questionnaires were 
prepared through a collaborative effort by an international team of experts, and 
extensively reviewed by the national research coordinators of the participating countries. 
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Four background questionnaires were developed to collect information at 
different levels of the system. The student questionnaire sought information about home 
characteristics, attitudes, and school experiences. The teacher questionnaires (one for the 
mathematics teacher and another for the science teacher) gathered information about 
preparation to teach, contents covered in classes, and instructional approaches. The 
school questionnaire focused on school staffing and facilities, curriculum, and 
instructional arrangements. The curriculum questionnaire addressed issues of curriculum 
design and policies (Mullis, Martin & Stemler, 2000). 

Most of the items in the questionnaires used a four-point Likert scale format 
(e.g., strongly agree , agree , disagree, and strongly disagree). The exception was the 
curriculum questionnaire - one per country - where respondents were asked to write 
explanations about the schooling system and mathematics curriculum in their 
jurisdictions. 



Exploratory Analyses 

Data Screening 

A crucial step in this dissertation study was the selection of the background 
variables that would be used to understand better the achievement differences between 
Chile and the comparison jurisdictions, as well as the achievement differences within 
Chile. Bearing in mind the research questions of this study, all the TIMSS 1999 
questionnaires were reviewed extensively for questions that could be used as indicators of 
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the constructs of interest. Once a preliminary set of items was selected, a standard 
checking procedure was implemented to verify the availability of the data in all five 
jurisdictions 17 , response rates, and the plausibility of the data. 

Univariate statistics were computed for each variable of interest (e.g., means and 
modes, ranges and variances). Outliers and skewed distributions were identified. Some 
variables were recoded to reflect better the distribution of responses among the available 
options. Categories were sometimes combined to avoid missing data. For instance, the 
original six response options of students’ educational expectations were recoded into 
three categories. The first category, “finish secondary school or less” (1 point), 
encompassed some secondary school and finish secondary school. The second category, 
“Some vocational/technical or some university” (2 points) encompassed some 
vocational/technical education after secondary school , some university , 7 don’t know (6% 
of total cases), and also the missing responses (20% of total cases). The last category, 
finish university (3 points), was kept in its original form. This procedure was equivalent 
to use mean imputation to replace missing and 7 don ’t know values. 

The bivariate association between the variables was investigated together with 
their relationship with mathematics achievement. Some variables were reverse scored so 
that the scale direction made more sense conceptually. 



17 Jurisdictions could omit/adapt items in the questionnaires that were not considered appropriate for 
their population. 
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Background Variables 



Table 3.3 presents the background variables used in this study. The variables were 
grouped in dimensions or blocks that reflected the broad conceptual categories addressed 
in the research questions. The blocks were: a) schools’ community contexts, 
b) implemented mathematics curriculum, c) teacher quality, d) school resources, e) type 
of school, f) students’ attitudes toward mathematics, and g) students’ family background. 

In total, 17 variables (single- and multiple-item indicators) were used to measure 
the different facets of these dimensions. For each variable, the table provides information 
about the source question(s) in the TIMSS 1999 questionnaires, category options and 
score points. In the case of multiple-item indicators, a brief explanation of how these 
derived variables were computed is also provided. Detailed information about the 
computation of multiple-item indicators is provided in the following section. 

The background variables presented in the table below were further transformed 
to fit the needs of different statistical analyses. The nature of these transformations is 
explained together with the corresponding analyses. 
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Table 3.3 



Detailed Information About Background Variables 



Variable 


Description 


SCHOOLS' COMMUNITY CONTEXTS 


Socio- 
economic 
level of the 
school 
community 


Index score computed by averaging the score points associated with the 
responses to father and mother education, number of books at home, and 
possessions at home (as reported by the students). All items were given equal 
weight and were aggregated at the school/class level. 

How far in the school did your father and mother go? (SQ_7) 

Response options: 

Some primary or did not go to school (0 point) 

Finished primary school (1 points) 

Some secondary school (2 points) 

Finished secondary school (3 points) 

Some vocational/technical education after secondary school (4 points) 
Some university (5 points) 

Finished university (6 points) 

About how many books are there in your home ? (SQ_1 0) 

Response options: 

None or very few (0 point) 

Enough to fill one shelf (1 points) 

Enough to fill one bookcase (2 points) 

Enough to fill two bookcases (3 points) 

Enough to fill three or more bookcases (4 points) 

Do you have any of these items at your home? (SQ_1 1 a-l) 

1. calculator 

2. computer 

3. study desk/table 

4. dictionary 

Response options: No (0 point) - Yes (1 point) 


Urban/rural 


In what type of community is your school located? (SCQ_1 ) 
Category options: 

Away from city center (0 point) - Closer to city center (1 point) 


Note. SCO = Schools’ questionnaire, SQ = Students' questionnaire. Item numbers match the 
general version of the questionnaires. 
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Table 3.3 



Detailed Information About Background Variables (cont.) 



Variable 


Description 


IMPLEMENTED MATHEMATICS CURRICULUM 


Content 
coverage 
(advanced) * 


Index score computed across 20 questions by averaging the score points 
associated with each response option. Topics were grouped in 4 content areas 
(measurement; geometry; algebra; and data representation, analysis and 
probability). Score points were weighted proportional to the number of 
mathematics problems from the advanced content areas in the TIMSS 
mathematics test. 




The following list includes the main topics addressed by the TIMSS mathematics 
test. Check the response that describes when students in your mathematics 
class have been taught each topic (list of 20 topics) 

(TQMB_1 3bl 3-c24, e27-f34) 

Response options; 

Taught before this year (1 point) 

Taught 1-5 periods this year (1 point) 

Taught more than 5 periods this year (2 points) 

Not yet taught (0 point) 

1 do not know (0.5 points) 


Content 

coverage 

(general)* 


Same as advanced index but including 14 additional questions about fractions, 
number sense, and probability topics. Score points were weighted proportional 
to the number of mathematics problems from each content area in the TIMSS 
mathematics test. (TQMB_13al-f34) 


Subject 

emphasis 


What subject matter do you emphasize most in your mathematics class? 
(TQMB_2) 

Category options: Basic content (whole numbers, fractions, decimals, 
percentages) (0 point) - Advanced content (geometry, algebra, or a 
combination of them with other subjects) (1 point) 


Additional 

mathematics 

courses 


Schools sometimes organize instruction differently for students with different 
abilities and interests in mathematics. Which of the following does your 
school do for students in the 8 th grade? (SCQ_20a-d) 

1 . Remedial mathematics is offered 

2. Enrichment mathematics is offered 

Category options: None (0 point) - Either remedial or enrichment (1 point) - 
Both remedial and enrichment (2 points) 


School 

curriculum 


Does your school have its own written statement of curriculum content to be 
taught for mathematics? (SCQ_10a) 

Response options: No (0 point) - Yes (1 point) 


Instructional 

materials 


Has your school developed instructional activities or learning materials to address 
the curriculum taught in your school? (SCQ_1 la) 

Response options: No (0 point) - Yes (1 point) 



Note. SQ = Students' questionnaire, SCQ = Schools' questionnaire, TQM = Mathematics teachers’ 
questionnaire (part A or B). Item numbers match the general version of the questionnaires. 

* The advanced content coverage index was used for the international comparisons only. Since 
this measure did not have enough variability in Chile, the general index was used for the national 
analyses. 
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Table 3.3 



Detailed Information About Background Variables (cont.) 



Variable 


Description 


TEACHER QUALITY 


Teacher 

qualification 


Index score computed across 4 questions by averaging the score points 
associated with each response category. Equal weight was given to each 
question. 




What is the highest level of formal education you have completed? (TQMA_1 5) 
Response options: Did not complete secondary school (0 point) - 
Secondary only (1 point) - Bachelor or equivalent (2 points) - 
Master or Doctorate (3 points) 

Do you have a teacher training certificate? (TQMA_1 6a) 

Response options: No (0 point) - Yes (1 point) 

While studying to obtain your teacher training certificate, what was your major or 
main area of study? (TQMA_1 7a,c,e,f) 

1. Mathematics 

2. Mathematics education 
Response options: No (0 point) - Yes (1 point) 

If you have a master's degree, what was your major or main area of study? 
(TQMA_18a,c,e,f) 

1. Mathematics 

2. Mathematics education 
Response options: No (0 point) - Yes (1 point) 


Preparation 
to teach 


Index score computed across 12 questions by averaging the score points 
associated with each response option. 




How well prepared do you feel you are to teach (TQMA_14a-l) 

1 . Fractions, decimals and percentages 

2. Ratios and proportions 

3. Measurement - units, instruments, accuracy 

4. Perimeter, area, and volume 

5. Geometric figures - definitions and properties 

6. Geometric figures - symmetry, motions, and transformations, 
congruence and similarity 

7. Coordinate symmetry 

8. Algebraic representation 

9. Evaluate and perform operations on algebraic expressions 

1 0. Solving linear equations and inequalities 

1 1 . Representation and interpretation of data in graphs, charts, and tables 

12. Simple probabilities - understanding and calculations 

Response options: Not well prepared (0 point) - Somewhat prepared (1 point) - 
Very well prepared [2 points) 



Note. SQ = Students' questionnaire, SCQ = Schools’ questionnaire, TQM = Mathematics teachers’ 
questionnaire (part A or B). Item numbers match the general version of the questionnaires. 
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Table 3.3 

Detailed Information About Background Variables (cont.) 

Variable Description 

SCHOOL RESOURCES 

Limitations Index score computed across 1 1 questions by averaging the score points 

in school associated with each response option. 

resources Is your school's capacity to provide instruction affected by a shortage or 

inadequacy of any of the following (SCQ_1 2a-e,g-k,r). 

1 . Instructional materials (e.g., textbooks) 

2. Budget for supplies (e.g., paper, pencils) 

3. School buildings and grounds 

4. Heating/cooling and lighting systems 

5. Instructional space (e.g., classrooms) 

6. Computers for mathematics instruction 

7. Computer software for mathematics instruction 

8. Calculators for mathematics instruction 

9. Library materials relevant to mathematics instruction 

1 0. Audio-visual resources for mathematics instruction 

1 1 . Teachers qualified to teach mathematics 
Response options: 

None (0 point) - A little (1 point) - Some (2 points) - A lot (3 points) 

SCHOOL ADMINISTRATION 

School School administration (MINEDUC) 

administration Category options: 

Public/municipal (0 point) - 

Private (elite-paid or private-subsidized) (1 point) 

Note. SQ = Students' questionnaire, SCO = Schools' questionnaire, TQM = Mathematics teachers’ 
questionnaire (part A or B). Item numbers match the general version of the Student Questionnaire. 
MINEDUC = Data provided by Ministry of Education, Chile. 
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Table 3.3 

Detailed Information About Background Variables (cont.) 

Variable Description 

STUDENTS’ ATTITUDES TOWARD MATHEMATICS 

Liking Index score computed across 4 questions by averaging the score points 

mathematics associated with each response option. 

How much do you like mathematics? (SQ_21 a) 

Response options: Dislike a Lot (0 point) - Dislike (1 point) - Like (2 points) - 
L like a Lot (3 points) 

I enjoy learning mathematics. (SQ_24a) 

I would like a job that involved using mathematics. (SQ_24e) 

Mathematics is boring / (SQ_24b) 

Importance Index score computed across 4 questions by averaging the score points 
of associated with each response option. 

mathematics* / think it is important to do well in mathematics at school. (SQ_1 5b) 

Mathematics is important to everyone's life. (SQ_24d) 

I need to do well in mathematics to get into the secondary school or university 
I prefer. (SQ_25c) 

I need to do well in mathematics to please myself. (SQ_25d) 

Difficulty Index score computed across 5 questions by averaging the score points 

of doing associated with each response option, 

mathematics / usually do well in mathematics. r (SQ_1 6a) 

Mathematics is an easy subject. r (SQ_24c) 

I would like mathematics much more if it were not so difficult. (SQ_1 7a) 

Although I do my best, mathematics is more difficult for me than 
for many of my classmates. (SQ_1 7b) 

Sometimes, when I do not understand a new topic in mathematics initially, 

I know that I will never really understand. (SQ_1 7d) 

Index score computed across 4 questions by averaging the score points 
associated with each response option. 

To do well in mathematics at school you need good luck. (SQ_1 8b) 

To do well in mathematics at school you need lots of innate talent. (SQ_1 8a) 
Nobody can be good in every subject, and I am just not talented for 
mathematics. (SQ_1 7c) 

Mathematics is not one of my strengths. (SQ_1 7e) 

Educational How far in school do you expect to go? (SQ_8) 
expectations Response options: Finish secondary school or less (0 point) - 

Some vocational/technical, some university, or don't know (1 point) - 

Finish university (2 points) 

Note. Unless otherwise indicated, all the questions had a 4-point Likert-format scale with response 
options Strongly Disagree (0 point), Disagree (1 point), Agree (2 points), and 
Strongly Agree (3 points). 

SQ = Students' questionnaire. Item numbers match the general version of the questionnaire. 
r Reversed scored scale. 

* For the national analyses, the variable “Importance of Mathematics” was dichotomized so that 
1 = Strongly agree to all four statements, and 0 = any other combination of responses. This 
transformation was implemented considering the highly skewed distribution of this index in Chile. 



Importance 
of luck & 
innate talent 
in doing 
mathematics 
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Table 3.3 

Detailed Information About Background Variables (cont.) 

Variable Description 

STUDENTS' FAMILY BACKGROUND 

Index score computed by averaging the score points associated with the 
responses to father and mother education, number of books at home, and 
possessions at home (as reported by the students). All items were given equal 
weight and were aggregated at the school/class level. 

How far in the school did your father and mother go? ISG 7) 

Response options: 

Some primary or did not go to school (0 point) 

Finished primary school (1 point) 

Some secondary school (2 points) 

Finished secondary school (3 points) 

Some vocational/technical education after secondary school (4 points) 

Some university (5 points) 

Finished university (6 points) 

About how many books are there in your home ? (SQ_1 0) 

Response options: 

None or very few (0 point) 

Enough to fill one shelf (1 point) 

Enough to fill one bookcase (2 points) 

Enough to fill two bookcases (3 points) 

Enough to fill three or more bookcases (4 points) 

Do you have any of these items at your home? (SQ_1 1 a-l) 

1. calculator 

2. computer 

3. study desk/table 

4. dictionary 

Response options: No (0 point) - Yes (1 point) 

Note. SQ = Students' questionnaire. Item numbers match the general version of the questionnaire. 



Computation of Multiple-Item Indicators 

The TIMSS 1999 questionnaires included sets of questions related to the same 
underlying construct. Whenever possible, these questions were combined to form indices. 
Indices present several conceptual and measurement advantages that make them 
preferable to single-item indicators. They are better suited to represent complex 
constructs difficult to capture in a single item, have greater reliability and discriminating 
power, allow for data reduction, and maximize the amount of valid cases (see DeVellis, 
1991; Spector, 1992; Sullivan & Feldman 1979). 



Socio- 
economic 
level of the 
students 
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A somewhat different procedure was used to compute different types of indices. 
The TIMSS 1999 dataset included some 40 variables that could be used to measure 
different facets of students’ attitudes toward mathematics. Principal component was used 
to identify the items that measured common underlying constructs (i.e., they loaded in the 
same underlying factor). Reliability analysis was used to assess the strength of the 
association among the items. 

In other cases, there were just two or three items available to measure the 
construct of interest. Chi-square and Spearman’s rank order correlation were used to 
measure the association between pair of variables before combining them. Finally, in 
some cases the source items were not expected to covary; these indices were more 
cumulative in nature. A good example is the offering of additional mathematics courses. 
In the Chilean schools, it was frequent to observe that neither remedial nor enrichment 
courses were available, or that the schools offered only remedial courses. The schools 
that offered enrichment courses only did so if remedial courses were available. 

In general, index scores were computed by averaging the score points associated 
with the response options of the selected questions. All the source variables were scaled 
so that more points corresponded to higher levels of the attribute being measured. In most 
of the cases the source variables were given equal weight to compute the index score. In 
general, two-thirds of the component variables had to have valid data for an index to be 
computed. 
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Imputation of Missing Data 



Preliminary analyses showed that, in Chile, almost 50% of the students had 
missing data for one or more of the variables shown in Table 3.3. The missing data posed 
a major problem at the school/class level, since all the students enrolled in those 
schools/classes were missing that information after disaggregating the data at the student 
level. Missing data also posed a major problem for the hierarchical linear modeling of 
mathematics achievement (see next section), because the available software does not 
accept missing data at level 2 (the school/class level). Since it was reasonable to presume 
that the data was not missing at random, losing cases with missing data was likely to 
introduce bias into the parameter estimates. 

Combining variables into indices drastically diminished the amount of missing 
data. In general, an index score was computed for each case provided that the number of 
variables with missing values was less than one-third. If an index score could not be 
computed, then conditional mean imputation was used to impute data to replace missing 
values. This procedure was also used to impute data on the single-item indicators that 
were used in the statistical analyses. 

Using conditional mean imputation, a separate regression model was specified for 
each variable with missing data. The regression model was chosen to reflect the type of 
data: a linear model was used for continuous data, logistic regression for binary data, and 
multinomial logit for categorical data. The regression models were estimated beginning 
with the variables with the least missing data, and proceeding next with the variables with 
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more missing data. Predicted scores and predicted groups were then used to replace the 
missing data (see Allison, 2001). This procedure was performed independently for each 
of the five jurisdictions. 

Variables with too many missing data, or for which the predictive models did not 
seem to be accurate enough, were not imputed. In fact, during the preliminary analyses, 
only the variables with a reasonably high response rate were kept for further analysis. By 
the same token, schools/classes with too many missing data were dropped from the 
analyses, as well as the students enrolled in those schools/classes. In Chile, three 
schools/classes and their students were dropped from all the analyses. As a consequence, 
182 out of 185 (98%) schools/classes were analyzed from the Chilean dataset. In Chile, 
the proportion of students with valid data (after imputation) varied around 98%, 
depending on the analysis. 

In Miami-Dade County Public Schools, missing data were a more critical issue. In 
this jurisdiction, data from the School Questionnaire was available for only 20 out of the 
25 schools (80% of the sampled schools). As a consequence, the comparative analyses 
were run with a restricted sample. Caution is recommended in interpreting the data from 
this district. 

Chapter 4 provides the percentage of valid cases for each separate analysis (see 
Tables 4. 1-4.5 and Figures 4. 1-4.4). In general, these percentages were very high, with 
almost all the analyses having over 90% of valid cases across the five jurisdictions. The 
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modeling of mathematics achievement in Chile included over 97% of the original sample 



of students. 



Data Analysis 

Comparative Analysis of the Schooling Systems 

Univariate Analyses 

The first phase of this study compared the Chilean educational system with the 
systems in South Korea, Malaysia, the Slovak Republic, and Miami-Dade County Public 
Schools. As stated in the research questions, the aim was to learn to what extent the 
Chilean educational system differed from the comparison jurisdictions in regards to five 
dimensions: 

1. Schools’ community context 

2. Implemented mathematics curriculum 

3. Teacher quality 

4. School resources 

5. Students’ attitudes toward mathematics 

Within each dimension, several background variables were analyzed using Chile 
as the reference group. In practice, this implied standardizing the variables and centering 
them so that Chile had a mean equal to zero and a standard deviation equal to one (M = 0, 
SD = 1). Means and standard errors were computed using the jackknife repeated 
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replication (JRR) method. This method provides approximately unbiased estimates of the 
sampling errors by taking into account the complex sampling design used in TIMSS 1999 
(i.e., it avoids the assumption of simple random sampling). The JACKGEN macro 
provided in the TIMSS 1999 database computes means and standard errors using the 
jackknife repeated replication method. Confidence intervals around the mean were 
computed to test the significant level of differences. 

These univariate comparative analyses were weighted with TOTWGT, one of the 
weighting variables provided in the TIMSS 1999 database. TOTWGT takes into account 
the differential probability of the students being selected in the sample, and reproduces 
the number of cases in the target population of each jurisdiction. Therefore, statistical 
analyses with TOTWGT provide information about the target population of interest, 
rather than about the sampled students. The students were always the unit of analysis, 
even when schools/classes variables were under scrutiny. 

Multivariate Analyses 

Theory suggests that socio-economic level, the nature of the implemented 
curriculum, teacher quality, and school resources are not independent of one another (see 
Chapter 2). These background factors are expected to covary. For this reason, it is 
desirable to include them together in a single analysis. Multivariate analysis of variance 
(MANOVA) was run with this purpose. 

MANOVA tested how different Chile was from the comparison jurisdictions on a 
combination of variables taken together. Only the most relevant variables (as shown in 
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the univariate analyses) were used as input for this analysis. These variables were: 
students’ socio-economic level, content coverage (advanced), preparation to teach, and 
importance of luck and innate talent. First, an omnibus analysis was run to see if there 
were overall differences between the jurisdictions on the combined set of variables. Then, 
planned comparison was used to contrast Chile against the four jurisdictions combined 
(Chile versus others). 

All the analyses were weighted to account for the differential probability of the 
students being selected in the sample. As indicated before, sample sizes varied widely 
from one jurisdiction to the other. In this particular analysis, it was desirable to have all 
five jurisdictions contributing equally to the final statistics, no matter what their sample 
size was. Because of this reason, the analyses were weighted with SENWGT, a weighting 
variable provided in the TIMSS 1999 dataset. Within each jurisdiction, the sum of 
SENWGT is 500, no matter what the actual sample size was. This weight is 
recommended when international estimates are sought and you do not want them to be 
affected by differential sample sizes, i.e., all countries are weighted equally. The analyses 
were run using the SPSS MANOVA codes. The students were the unit of analysis, even 
though schools/classes variables were under scrutiny. 

Modeling Mathematics Achievement in Chile 

The second phase of this study aimed to discover if the background variables 
already identified were related to mathematics achievement in Chile. Mathematics 
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achievement was modeled at the school/class level, and at the student-within-class level 



using a hierarchical linear models. 

As a result of the TIMSS 1999 sampling design, students were nested within 
schools/classes (clusters). Therefore, students from the same schools/classes were 
expected to be more alike than students randomly sampled from the whole population of 
8 th graders. This was because these students shared more common experiences 
(e.g., same teacher, school resources) than students randomly drawn from different 
classes and schools. 

Considering the clustered structure of the TIMSS 1999 data set, a multilevel 
approach was adopted in this study. Multilevel modeling presents several advantages, 
both conceptual and technical, over ordinary least squares (OLS) regression. Hierarchical 
linear models (HLM) explicitly recognize the presence of hierarchical units of analysis 
and allow modeling them simultaneously, in interrelated sub-models (Hox, 2002; Heck & 
Thomas, 2000; Raudenbush & Bryk, 2002). 

The Hierarchical Linear Models 

This dissertation study stated several hypotheses about the relationship between 
background variables and mathematics achievement in Chile. These hypotheses were 
tested using a Two-level random intercept model. At the school/class level, the mean 
mathematics score of the schools/classes were modeled using community, curriculum, 
teacher, and school variables. At the students-within-class level, students’ mathematics 
scores served as the outcome and students’ characteristics as the predictors. 
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Before modeling mathematics achievement, it was important to know the 



proportion of achievement variance at the school/class level, and the proportion at the 
students-within-class level. The proportion of outcome variance at the school/class level 
set the upper-limit that could be explained by the models of mathematics achievement at 
the school/class level. By the same token, the proportion of outcome variance at the 
students-within-class level set the upper-limit that could be explained by the models of 
mathematics achievement at the students-within-class level. 

The between schools/classes (unconditional) model was the preliminary step 
before modeling mathematics achievement. Then, five schools/classes (conditional) 
models were tested to evaluate the partial contribution of different sets of predictor 
variables. Lastly, two students-within-class models were used to test the partial effect of 
students’ attitudes on achievement. Detailed explanations of these models are provided 
below. 



The between schools/classes model. This model estimated the ratio of school/class 
mean variance to the total achievement variance. This model is similar to a one-way 
analysis of variance because the variation in the dependent variable (mathematics 
achievement) is partitioned into two components: between schools/classes and students- 
within-classes. No predictors were entered at the school/class level, nor at the students- 
within-class level (Equations 1 and 2 respectively). 
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yv=0oj + r u 



(i) 



A), — Too +U oj 



( 2 ) 



Where: 

yij = Mathematics score of student i in school/class j 
J3 0j = Mean mathematics score for school/class j (intercept) 

y m = Grand mean mathematics score 

r,j = Random error associated with student i in school/class j 
mq j = Random error associated with school/class j 



The ratio of the school/class mean variance to the total achievement variance was 
computed as follows (Equation 3): 




7 00 

+ & 2 



(3) 



Where: 

= Variance of the school/class means around the grand mean 
(between-group variability) 

<7 2 = Variance of the students-within-school/class (within-group variability) 
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The school/class models. Five models of mathematics achievement were tested at 



the school/class level. The models were inclusive, with each one encompassing additional 
blocks of predictor variables. The blocks matched the grouping categories already 
presented in Table 3.3: the implemented mathematics curriculum, teacher quality, school 
resources, school community context, and school administration. Using inclusive models 
allowed estimating the proportion of additional outcome variance accounted for by the 
successive models. 

The use of the models was an attempt to see the effect of educational variables 
first, without considering the effect of socio-economic level. The indicators of curriculum 
implementation were entered first, in Model 1. This was because the implemented 
curriculum more directly conditions students’ opportunities to learn mathematics. 
Teacher quality indicators were entered second, in Model 2. This was because quality 
teachers are a necessary but not sufficient condition to produce quality learning. A good 
teacher also needs a good curriculum to boost achievement. Schools resources affect 
instruction in a more indirect way; hence this measure was included in a third model. The 
broader socio-economic context in which the school operates is a given condition, 
beyond the control of the school. The community socio-economic level was included in 
Model 4. Model 5 was restricted to the variables that had a more consistently significant 
effect. All these steps are summarized below: 
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Model 1 used five variables measuring the implemented mathematics 
curriculum. 



Model 2 encompassed the previous model, plus two variables measuring 
teacher quality. 

Model 3 encompassed the previous model, plus a variable measuring 
limitations in school resources. 

Model 4 encompassed the previous model, plus two variables measuring 
schools’ community contexts. 

Model 5 was restricted to the variables that had a significant partial effect on 
the outcome, conditional on the average socio-economic level of the 
community served by the school. This model also included a variable 
measuring the type of school administration (public/private). 

An additional model using only the socio-economic index was also tested. This 
sixth model allowed estimating how much additional variance was accounted for by 
Model 5, compared to a model that used the socio-economic index as predictor. 

Equation 4 presents the general formula for the school/class sub-models: 



fioj = Too + To i (comm j ) + y Q2 ( C urr j ) + y m (teach ,. ) + y 04 (school j ) ■ + u oj (4) 

Where: 

J3 aj = Mean mathematics score for school/class j (intercept) 

comnij = Value of a “schools’ community contexts” variable for school/class j 



92 




currj = Value of an “implemented mathematics curriculum” variable for school/class j 



teach j = Value of a “teacher quality” variable for school/class j 
school] = Value of “limitation in school resources” for school/class j 
y 00 = Grand mean mathematics score 

y 0l = Partial effect of the “schools’ community contexts” variable 

y Q2 = Partial effect of the “implemented mathematics curriculum” variable 

y 03 = Partial effect of the “teacher quality” variable 

y 04 = Partial effect of the “limitations in school resources” variable 

uoj = Random error associated with school/class j 



The students-within-school/class models. At the students-within-school/class 
level, one sub-model, including five attitude variables, was tested: liking mathematics, 
importance of mathematics, difficulty of doing well in mathematics, importance of luck 
and innate talent in doing mathematics, and educational expectations. Since all of the 
variables tapped one construct (i.e., attitudes toward mathematics), no successive blocks 
of variables were used. Instead, the five predictors were entered at once, as shown in 
Equation 5: 

y„=A,+P l {UKE f )+/3 2 {lMP, l )+P 1 {DIFF, l )+ 

A,(i UCK ll )+0,(EDEX^+r n 
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Where: 



/ 3 0j = Mean mathematics score for school/class j (intercept) 

j3 5 = Partial regression coefficients for five predictor variables 



LIKE,,... EDEXy 


= Predictor values for student i in school/classy (group-mean centered) 


LIKE 


= Liking mathematics 


IMP 


= Importance of mathematics 


DIFF 


= Difficulty in doing mathematics 


LUCK 


= Importance of luck and innate talent in doing mathematics 


EDEX 


= Educational expectations 



r,y = Random error associated with student i in school/class / 



A second students-within-school/class sub-model was run to find out if the partial 
effect of the attitudes variables was affected by the students’ socio-economic level. 

In the hierarchical linear analyses, all predictors other than dummy variables were 
standardized to facilitate the interpretation of results. At the students-within-class level, 
the predictors were group-mean centered. This allowed for the study of the relationship 
between attitudes and mathematics achievement within the schools/classes. 

Mathematics achievement is represented in the TIMSS 1999 database by five 
“plausible values” for each student. The five plausible values were entered in the 
plausible value function of the HLM 5.02 software (Raudenbush, Bryk, & Congdon, 
2000). The models were run using restricted maximum likelihood. This method was 
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chosen because it introduces less bias in the parameter estimates than maximum 
likelihood (Hox, 2002, p. 38). Robust standard errors were reported because they are less 
dependent on the normality assumptions (Hox, 2002, p. 201). All of the analyses were 
weighted by HOUWGT to account for the differential probability of the students being 
selected in the sample. This weighting variable reproduced the country sample size and 
was provided in the TIMSS 1999 dataset. 
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CHAPTER 4 



RESULTS 

This chapter answers the research questions of this dissertation study in two main 
steps. First, key characteristics of the Chilean educational systems were compared to the 
characteristics existing in the four comparison jurisdictions. Second, the relationship 
between these characteristics and mathematics achievement in Chile was further 
examined. 



Comparative Analysis of the Schooling Systems 

This section compares some key characteristics of the Chilean educational system 
with the characteristics of the educational systems in the four jurisdictions. The analyses 
focused on: a) schools’ community contexts, b) implemented mathematics curriculum, 
c) teacher quality, d) school resources, and e) students’ attitudes toward mathematics. 

How Different Was Chile from the Comparison Jurisdictions in 
Schools’ Community Contexts? 

Despite the fact that Chile shares a somewhat similar level of economic 
development with South Korea, Malaysia, the Slovak Republic, and Miami-Dade County, 
it was hypothesized that Chilean students came from a social background that was more 
disadvantaged. This hypothesis was tested using background data from the students who 
provided information about their parents’ education, availability of educational resources 
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at home (calculator, computer, study desk/table, and dictionary), and the number of books 
they had at home. This information was summarized in a socio-economic index. 



The socio-economic index was standardized so that Chile had a mean equal to 
zero and a standard deviation equal to the unit ( M = 0, SI) = 1). The jurisdiction means 
were recomputed in standard deviation scores from the Chilean mean. As shown in Table 
4.1, Chile had the lowest score in the socio-economic index when compared to the four 
jurisdictions. South Korea, the Slovak Republic and Miami-Dade County Public Schools 
scored three-fourths of a standard deviation above Chile. Chilean 8 th graders had a socio- 
economic index significantly lower than the students across the four jurisdictions 
( p < .005, one-tailed test). While Chilean students also scored significantly lower than the 
Malaysian students in the socio-economic index the gap was considerably smaller. 



Table 4.1 

Descriptive Statistics for Socio-Economic Level and Mathematics Achievement 



Socio-economic level 



M (SE) 



SD 



% Variation 
among schools* 



Mathematics 
achievement 
M (SE) 



Chile 


0 . 


. 00 


(0. 


. 05) 


1 . 


. 00 


45% 


392 


(4. 


.5) 


South Korea 


0 . 


. 72 


(0. 


. 02) 


0 . 


. 85 


16% 


587 


(2. 


.0) 


Malaysia 


0 . 


. 14 


(0. 


. 04) 


0 . 


. 88 


33% 


520 


(4. 


.2) 


Slovak Republic 


0 . 


. 73 


(0. 


. 03) 


0 . 


. 74 


26% 


534 


(3. 


.9) 


Miami-Dade County 


0 . 


. 72 


(0. 


.07) 


0 . 


.87 


13% 


421 


(9. 


.5) 


Public Schools 























Note. Valid cases across the five jurisdictions = 99.6%. Analysis weighted by TOTWGT. 

* Computed as the ratio of sum of squares between to sum of squares total (SSb / SSt). 
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At the country level (i.e., excluding Miami-Dade County), the socio-economic 
index correlated with average mathematics achievement. Higher performing South Korea 
and the Slovak Republic also had the higher socio-economic index, whereas middle 
performing Malaysia had a lower index level. Chile had both the lowest index level and 
the lowest average mathematics achievement. 

In Table 4.1, the standard deviation column indicates how much the students from 
each jurisdiction differed in their socio-economic status. Chilean students had the highest 
socio-economic spread ( SD = 1.00) while the Slovakian the least ( SD = 0.74). These 
results indicate that, on average, Chilean students were more heterogeneous in their 
socio-economic backgrounds than students from other jurisdictions. 

Table 4.1 also shows the socio-economic dispersion among the schools for each 
jurisdiction. The higher the percentage of index variance among the schools, the higher 
the levels of socio-economic segregation in that educational system. Countrywise, Chile 
had the highest proportion of index variance among the schools (45%), followed by 
Malaysia (33%), the Slovak Republic (26%), and South Korea (16%). It is interesting to 
note that, among the countries, the proportion of index variance among the schools was 
inversely correlated with average mathematics achievement. South Korea, the highest 
performing jurisdiction, had the least economic segregation in the school system. Chile, 
the lowest performing jurisdiction, had the most. 

Miami-Dade County Public Schools had the least socio-economic variation 
among the schools. There is evidence that the United States has one of the most 
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economically segregated educational systems in the world (Beaton & O’Dwyer, 2002), 
and Miami-Dade has a large percentage of students (59%) from low-income families - 
based on the percentage of students eligible to receive free or reduced-price lunch 
(Thornton & Wongbundhit, 2002). 

Furthermore Figure 4.1 breaks down the socio-economic index in its source 
variables, which were also standardized and centered on the Chilean mean, so that in 
Chile M = 0 and SD = 1. The y-axis indicates deviations (in standard scores) from the 
Chilean mean. Results indicate that Chilean students differed from the students in other 
jurisdictions by having less educated parents (both father and mother), fewer books, and 
fewer educational resources at home. In all four measures, the Chilean mean was 
significantly lower than the mean of the four jurisdictions combined (p < .005, one-tailed 
test). 



The source variable that differed the most from the Chilean average was the level 
of education reached by the students’ mothers. In Miami-Dade County, the average 
educational level of mothers was more than a standard deviation above the Chilean mean, 
whereas in the Slovak Republic it reached three-fourths of a standard deviation. While 
Chilean students were more like their peers in Malaysia regarding their mothers’ 
education, substantial differences still arose. In fact, while 37% of Chilean students 
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reported that their mothers at least finished secondary school, this proportion reached 
50% in Malaysia. 18 



Figure 4.1 

Differences in Socio-Economic Indicators 
Between Chile and the Comparison Jurisdictions 




Note. CHL = Chile (M = 0, SD = 1 ), MYS = Malaysia, SVK = Slovak Republic, KOR = South Korea, 
MIA = Miami-Dade County Public Schools. Valid cases by column: 80%, 76%, 99%, and 99%. 
Analysis weighted with TOTWGT. 



These results confirm the hypothesis that, despite sharing similar global indicators 
in economic development, Chile has a serious disadvantage in comparison to the other 
four jurisdictions used in this study. The cultural capital of the students’ families was 
substantially lesser in Chile than in South Korea, Malaysia, the Slovak Republic, and 
Miami-Dade County. Most notably, Chilean students’ parents had considerably fewer 
years of schooling than parents in other jurisdictions. This is an important finding since, 



18 Percentages based on all valid responses other than I don ’t know. 
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among all the background variables collected by TTMSS 1999, mothers’ and fathers’ 
education were probably the strongest predictors of students’ mathematics achievement. 

South Korea, the Slovak Republic, and the United States had universal coverage 
of primary education in the 1960s; Malaysia did so in the 1980s. In Chile, only a tiny 
elite attended the school in the 1960s, and universal coverage was reached in the 1990s. 
These results emphasize the importance of raising the educational level of Chile’s future 
generations. Steps are being taken to respond to Chilean educational deficits. Access to 
education has been rising steadily during the last decades. A 2004 government report 
stated that over 90% of the young were enrolled in school (MIDEPLAN, 2004). It is 
likely that greater access to education and increased years of schooling will have a long 
lasting ripple effect in future families of the current high school students. However, this 
positive trend will only show up in some 20 more years. 

Beyond the socio-economic level, the community contexts in which the schools 
operate may differ in their urban/rural location. Schools in urban settings are more likely 
to have access to technical and material resources, therefore it is important to know if the 
schools in Chile differed from the schools in the other jurisdictions in regard to this. In 
Chile, 80% of the students attended a school close to the center of a town/city or on the 
outskirts of a town/city. In South Korea and Miami-Dade County Public Schools, more 
students were enrolled in urban schools (88% and 100%, respectively). In Malaysia and 
the Slovak Republic, substantially fewer students were enrolled in this type of school 
(59% and 67%, respectively). This result shows that Chile did not stand apart from the 
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other jurisdictions in the urbanization of its schools. This finding does not support the 
hypothesis that the Chilean schools were less urban than the schools in the other 
jurisdictions. 

How Different Was Chile from the Comparison Jurisdictions in the 
Implementation of the Mathematics Curriculum? 

The mathematics curriculum (as it is implemented in the schools) influences the 
opportunities students have to learn different content and skills. In this dissertation study 
it was hypothesized that Chilean schools had poorer strategies to implement the official 
mathematics curriculum when compared to the schools in other jurisdictions. It was also 
hypothesized that Chilean students had fewer opportunities to learn the most advanced 
mathematics content assessed in the TIMSS 1999 test when compared to the students 
across the other jurisdictions. Five measures of curriculum implementation were used to 
test these hypotheses: 

1. The schools having their own written statement of the mathematics content to 
be taught (other than the national or regional curriculum guides). 

2. The schools having developed instructional activities or learning materials to 
address the mathematics curriculum. 

3. Index of content coverage (advanced), based on four content areas assessed in 
the TIMSS 1999 mathematics test (measurement; geometry; algebra; and data 
representation, analysis and probability). 
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4. Emphasis on subject areas: basic content (whole numbers, fractions, decimals, 
percentages) versus advanced content (geometry, algebra, or a combination of 
them with other subjects). 

5. The schools providing additional mathematics courses to address the needs of 
students with different levels of knowledge and skills - remedial and/or 
enrichment courses 

Figure 4.2 shows how the four comparison jurisdictions differed from Chile in 
each one of these measures. Here again, deviations from zero (the Chilean mean) were 
standardized so that they could be interpreted in the context of the Chilean distribution. 
Results indicate that Chile was lagging in terms of the implementation of the 
mathematics curriculum. 

Chilean schools differed from the comparison jurisdictions in their development 
of curricular guides, materials and activities to support the teaching of mathematics. In 
Chile, 16% of the students were enrolled in schools with their own mathematics 
curriculum. Despite having more detailed and demanding official curricula than Chile, in 
three jurisdictions this proportion was substantially higher: South Korea (66%), Miami- 
Dade County Public Schools (59%), and Malaysia (47%). The Slovak Republic was the 
exception, with only 8% of the students enrolled in schools that had their own 
mathematics curriculum. However, as noted previously, the Slovak Republic has a very 
detailed national curriculum used for all students. 
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Figure 4.2 

Differences in the Implemented Mathematics Curriculum 
Between Chile and the Comparison Jurisdictions 



C-.KJKJ ~ 

1.50 - 
1.00 - 
0.50 - 

. = 0 


< 

> 

< 


> 

< 

> 

r 


! I > 

) J o E 


» 

< 

1 


0.50 - 
1.00 - 
1 SD 


V 

1 


* < 

1 


> 

1 < 


J 

> 


1 .\JU 


School with 
own math 
curriculum 


School with 
own math 
materials 


Content 

coverage 

(advanced) 


Emphasis 
on advanced 
contents 


Additional 

math 

courses 


KOR 


1.82 


-0.38 


1.01 


1.50 


-0.93 


MIA 


1.16 


0.24 


0.69 


1.20 


0.50 


MYS 


0.83 


0.63 


0.44 


0.88 


1.08 


CHL 


0.00 


0.00 


0.00 


0.00 


0.00 


SVK 


-0.24 


-0.85 


0.55 


1.36 


0.12 



O 


KOR 


X 


MIA 


• 


MYS 


o 


CHL 


■ 


SVK 



Note. CHL = Chile (M = 0, SD = 1 ), MYS = Malaysia, SVK = Slovak Republic, KOR = South Korea, 

MIA = Miami-Dade County Public Schools. Overall response rates by column: 88%, 94%, 95%, 94%, 
and 99%. Analysis weighted with TOTWGT. 



Regarding the development of instructional materials and activities by the 
schools, findings varied. Sixty eight percent of Chilean students attended schools that had 
their own materials/activities. This proportion was substantially higher in Malaysia and 
Miami-Dade County Public Schools (98% and 80%, respectively) and lower in South 
Korea and the Slovak Republic (51% and 29%, respectively). 



Chilean 8 th graders were taught significantly less content than the students across 
the four jurisdictions, as shown by the index of advanced content coverage (p < .005, 
one-tailed test). It is notable that South Korea scored one standard deviation above Chile 
in this index. 
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The gap in the type of mathematics taught could also be seen in the content most 
emphasized by the teachers in their mathematics classes. In Chile, 73% of the students 
were taught by teachers who emphasized “mainly numbers.” In the comparison 
jurisdictions, this proportion was substantially smaller: 6% in South Korea, 12% in the 
Slovak Republic, 19% in Miami-Dade County Public Schools, and 33% in Malaysia. 

From these analyses it is clear that Chilean teachers were not teaching the 
mathematical content deemed important by the international community. While some of 
these content were not part of the Chilean curriculum (e.g., rounding), others were not 
covered despite being part of it (e.g., symmetry and transformations). The lack of rigor 
and detail in the Chilean curriculum is partly to blame for this poor coverage. The 
inability of the Chilean schools to produce their own mathematics curriculum and 
instructional resources probably further contributed to the disparity in students’ 
opportunities to learn mathematics. 

Finally, the implemented mathematics curriculum was also evaluated by the 
opportunities the students had to take remedial and/or enrichment mathematics courses in 
their schools. In Chile, 28% of the students attended a school offering both remedial and 
enrichment mathematics, compared to 87% in Malaysia, 45% in Miami-Dade Public 
Schools, and 30% in the Slovak Republic. Only in South Korea there were fewer students 
than in Chile enrolled in schools that offered both courses (12%). 

It is interesting to note the vast differences existing in the provision of additional 
mathematics courses in Chile. While 83% of the students were enrolled in schools 
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offering remedial mathematics, only 29% were in schools offering enrichment courses. 
This is probably so because the learning needs of low-achievement students are given 
priority over the needs of their high-achievement mates in the Chilean schools. 

It seems that Chilean students have few opportunities to learn advanced 
mathematics content. Teachers barely cover this material in their regular classes, and 
advanced mathematics courses are not offered in most of the schools. It is likely that the 
higher performing students of each school are not exposed to challenging enough classes. 
This situation deserves more attention from educational authorities, since previous 
research has shown that students in the same classes cover a wide spread of mathematics 
knowledge and skills (Ramirez, 2003). 

It is notable that South Korea, the highest performing jurisdiction, outperformed 
the other jurisdictions in three out of five measures of curriculum implementation. These 
measures were: coverage of advanced mathematics content, emphasis on advanced 
contents, and schools having developed their own curriculum. This evidence supports the 
notion that a positive relationship exists between curriculum implementation and 
mathematics performance. 
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How Different Was Chile from the Comparison Jurisdictions 
in Teacher Quality? 

In this dissertation study, it was hypothesized that Chilean students were taught 
by less qualified mathematics teachers than the students in the comparison jurisdictions. 
To test this hypothesis, an index of teacher qualification was computed by combining 
information from teachers’ educational level, training certificate, and mathematics major. 

As shown in Figure 4.3, the students in Chile were taught by less qualified 
teachers than the students across Miami-Dade County Public Schools, the Slovak 
Republic, and South Korea (p < .005, one-tailed test). While just 1% of the students in 
Chile had a teacher with a master or Ph.D. degree, this percentage increased to 15% in 
South Korea, 56% in Miami-Dade County Public Schools, and 97% in the Slovak 
Republic. In practice, Slovak teachers must have a graduate university diploma to teach 
(J. Kuraj, Ministry of Education Slovak Republic, personal communication, October 25, 
2004). Teachers usually get their first degree in a subject area (e.g., mathematics) and 
then complete a master or Ph.D. in education (Vantuch, 1995). 

Chilean students were taught by more qualified teachers than Malaysian students 
( p < .005, one-tailed test). This difference was largely because 19% of the Malaysian 
students were taught by teachers with a diploma from a technical/vocational institution. 
In Chile, teachers were required to have a university degree to teach; therefore almost all 
students (98%) were taught by teachers with a university degree. 
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Figure 4.3 

Differences in Teacher Quality 

Between Chile and the Comparison Jurisdictions 




Note. CHL = Chile (/VI = 0, SD = 1 ), MYS = Malaysia, SVK = Slovak Republic, KOR = South Korea, 

MIA = Miami-Dade County Public Schools. Overall response rates by column: 99% and 99%. 

Analysis weighted with TOTWGT. 

These findings partially support the hypothesis that Chilean students were taught 
by less qualified teachers than students in the comparison jurisdictions. It is unclear 
whether the demands of the degree required to teach in Chile were equivalent to those of 
Malaysia. The vocational/technical title obtained by teachers in Malaysia may in fact be 
superior than the training obtained in Chilean universities. For comparison purposes it 
would be helpful to have data on the quality and demands of the training programs 
instead of only having the degree title. 



The relationship between average mathematics achievement and teacher 
qualification was not clear. Miami-Dade County teachers were among the most qualified 
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and yet their students’ mathematics performance was very poor. In contrast, South Korea 
had the highest mathematics scores of the jurisdictions but its teachers did not have the 
highest qualifications. 

Another facet of teacher quality is teachers’ preparation to teach a range of 
mathematics content. In this dissertation study it was hypothesized that teachers were less 
prepared to teach mathematics in Chile. This hypothesis was tested using an index of 
preparation to teach. Teachers evaluated how prepared they felt in a list of mathematics 
content. The response options were not well prepared = 0 point, somewhat prepared = 1 
point, and very well prepared = 2 points. 

The average index score in Chile was M = 1.32. This value indicates that Chilean 
8 th graders were taught by teachers who felt somewhat prepared to instruct relevant 
mathematics content. In contrast, the students in the four comparison jurisdictions were 
taught by teachers who felt significantly better prepared to teach the contents ( M = 1.67, 
p < .005, one-tailed test). The average index score for each jurisdiction was: the Slovak 
Republic M = 1.88, Miami-Dade County Public Schools M = 1.80, Malaysia M = 1.79, 
and South Korea M = 1.56. The data supports the hypothesis that Chilean students were 
taught by teachers that were less prepared than students in the comparison jurisdictions. 
Figure 4.3 shows the relative standing of Chile in teacher preparation, after standardizing 
and centering the index score in the Chilean mean. 

The relationship between the index of teachers’ preparation to teach and average 
mathematics achievement was at times contradictory. Teachers in the top performing 
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South Korea did not feel better prepared to teach than the teachers in the bottom 
performing Miami-Dade County Public Schools. This counter-intuitive finding may be 
the result of Korean teachers evaluating their ability to teach the extremely demanding 
mathematics curriculum that South Korea uses. An additional explanation is that the 
responses given in a self-report survey like TIMSS may be influenced by cultural 
differences. It seems that Asian respondents tend to under-rate their knowledge and skills, 
whereas more westernized respondents tend to over-rate them. 

How Different Was Chile from the Comparison Jurisdictions in 
School Resources? 

Shortages and inadequacies in school resources may affect instruction, and this in 
turn may affect students’ mathematics achievement. In this dissertation study it was 
hypothesized that more students in Chile were enrolled in schools whose capacity to 
provide instruction was affected by scarcity of resources. This hypothesis was tested 
using an index of limitation of school resources. This index was based on the school 
principals’ perceptions of the extent to which shortages or inadequacies in a list of 11 
resources (e.g., library, computers) affected the schools’ capacity to provide instruction. 
The response options were none = 0 point, a little =1 point, some = 2 points, and a lot = 3 
points. 19 



19 The percentage of valid cases for the index of school resources was 99% (across all five jurisdictions). 
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In Chile, the average index score was M = 1.20, which was near a little. This 
result suggests that limitations in resources had a minor effect on the capacity of the 
Chilean schools to provide instruction. Three jurisdictions reported significantly more 
resource limitations than Chile (p < .05). These were Malaysia, M = 1.29; the Slovak 
Republic, M = 1.33; and South Korea, M- 1.51. Miami-Dade County Public Schools had 
principals that reported fewer limitations, M = 1.15; the difference between Chile and 
Miami-Dade County was not significant. 

These findings do not support the hypothesis that Chilean schools may be affected 
by more resource limitations than the schools in the comparison jurisdictions. To the 
contrary, Chilean schools seem to have more resources than schools in the other 
jurisdictions, with the exception of Miami-Dade County Public Schools. 

In interpreting these results, it is important to bear in mind that the index of 
limitations of school resources is a subjective measure based on the principals’ 
perceptions instead of on the objective physical conditions of the schools. This is a 
limitation since the principals’ perceptions are probably affected by their own 
expectations of what the school resources should be. In fact, it is in South Korea where 
the principals perceived the most limitations due to scarce/inadequate resources. This 
stringent evaluation is likely the result of the high standards that characterize the Asian 
culture, rather than the actual school conditions. 

To understand Chile’s positive evaluation regarding school resources one needs to 
take into consideration the time and context. It is possible that the positive evaluation of 
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school resources was a response to the reform efforts that have taken place during the last 
decade. Many Chilean schools have recently benefited from the construction of new 
buildings and libraries, expansion of grounds, and addition of computers. Principals may 
still be astonished with the improved conditions of their schools, and tend to be optimistic 
and generous in their evaluations. 

How Different Was Chile from the Comparison Jurisdictions in 
Students' Attitudes Toward Mathematics? 

In this dissertation study, it was hypothesized that Chilean students had more 
negative attitudes toward mathematics than students in the comparison jurisdictions. To 
test this hypothesis, this study used five indicators of students’ attitudes toward 
mathematics. Each indicator was aimed to target a different facet of the attitude construct. 
These indicators were: 

1. Importance of mathematics 

2. Importance of luck and innate talent in doing mathematics 

3. Difficulty in doing mathematics 

4. Liking mathematics 

5. Educational expectations 

The five measures were standardized and centered so that in Chile M = 0 and 
SD = 1. As shown in Figure 4.4, Chilean 8 th graders reported that mathematics were more 
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important for them in comparison to students from the other four jurisdictions ( p < .01, 
two-tailed test). 



Figure 4.4 

Differences in Students ’ Attitudes Toward Mathematics 
Between Chile and the Comparison Jurisdictions 




Note. CHL = Chile (A/I = 0, SD = 1 ), MYS = Malaysia, MIA = Miami-Dade County Public Schools, 
SVK = Slovak Republic, KOR = South Korea. Overall response rates by column: 100%, 99%, 100%, 
99%, and 100%. Analysis weighted with TOTWGT. 



South Korea, the jurisdiction with the highest performance in mathematics, had 
the lowest value in the index of importance of mathematics. The low relative standing of 
this country should not be interpreted as Korean students not valuing mathematics. On 
average, Korean students agreed to all the questions about importance of mathematics. 
However, students in the other jurisdictions were more likely to strongly agree with the 
same questions. 
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A measurement problem may be obscuring the “real” perceived importance of 
mathematics in Chile. It seems that in low performing countries students tend to over- 
report the value of mathematics. Little is known about this phenomenon, but one 
plausible reason is that these students tend to compensate their “failure” in the 
achievement tests by answering in a socially desirable way to the questionnaires. Shen 
and Pedulla (2000) reported a similar phenomenon in their analysis of students’ attitudes 
in several countries. 

As expected, Chilean students reported stronger beliefs in the importance of luck 
and innate talent in doing mathematics when compared to students across other four 
jurisdictions ( p < .01, two-tailed test). There is a strong body of research showing that 
students that place more emphasis on factors that are outside their reach (e.g., luck, innate 
intelligence) have lower academic performance than those who place more emphasis on 
factors such as effort and motivation (Benham, 1995). Effective teacher feedback is 
considered a key variable for promoting stronger beliefs in the importance of effort and 
motivation. The pervasive grade inflation existing in Chile (see Riveros, 2004) is 
probably concealing serious failures in teachers’ feedback. Perhaps fewer teachers in 
Chile were providing effective feedback to their students in comparison to teachers in the 
four jurisdictions. 

Students in Chile reported that mathematics were more difficult for them than 
students in Malaysia, Miami-Dade County Public Schools, and the Slovak Republic 
( p < .003, two-tailed test). However, it was in South Korea, the highest performing 
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jurisdiction, where the students reported that mathematics were the hardest. Korean 
students scored significantly higher than Chilean students on the index of difficulty in 
doing mathematics (p < .05). This is probably the consequence of the highly demanding 
curriculum of South Korea. 

In Chile, three-fourths of the students endorsed the statements “I like 
mathematics” and “I enjoy learning mathematics.” Chilean students had significantly 
higher scores in the liking mathematics index when compared to the students across 
South Korea, the Slovak Republic, and Miami-Dade County Public Schools (p < .003, 
two-tailed test). However, Chilean students scored significantly lower than Malaysian 
students in this index (p < .003, two-tailed test). This was an unexpected finding. This 
dissertation study hypothesized that the low mathematics performance of Chilean 
students was due, at least in part, to the students not liking the subject matter. More 
specifically, it was hypothesized that Chilean students liked mathematics less than 
students in the other four comparison jurisdictions. 

In Chile, 43% of the 8 th graders planned to graduate from a university; another 
40% planned to study in a vocational/technical institution after secondary school. These 
were very high expectations for a country where only half of the adults have finished 
secondary school (MIDEPLAN, 2004). Nevertheless, students’ expectations in Chile 
were significantly lower than across South Korea, Miami-Dade County Public Schools, 
and Malaysia (p < .003, two-tailed test). Only students in the Slovak Republic reported 
significantly lower educational expectations than students in Chile (p < .05, two-tailed 
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test). These results partially support the hypothesis that Chilean students may have lower 
expectations for further education, compared to students in the other jurisdictions. 

Summarizing, it is unclear whether Chilean students had more negative attitudes 
toward mathematics when compared to the students in other jurisdictions. While some 
facets of the attitudes construct confirm the research hypothesis, others clearly do not. 

How Different Was Chile from the Comparison Jurisdictions 
in a Combination of Background Variables Taken Together? 

In the previous section, separate comparative analyses were presented for five 
dimensions related to mathematics achievement. These were: a) the socio-economic level 
of the students’ families, b) the mathematics curriculum as it was implemented in the 
schools, c) teacher quality, d) school resources, and e) students’ attitudes toward 
mathematics. The variables used as indicators of these dimensions were expected to 
covary. For instance, higher socio-economic levels are usually associated with more 
qualified and better prepared teachers, more school resources, and with students’ that 
have more positive attitudes. Since these background variables are not always 
independent of one another, this section considered them together. 

This section addresses the following question: Is the combination of background 
variables different for Chile than for the comparison jurisdictions? It was hypothesized 
that the answer would be yes. Only a subset of the variables already examined was 
selected for the combined analyses. Criteria for selecting these variables were that they 
must represent different dimensions of the educational process, and that they had shown 
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significant differences among the jurisdictions in the previous analyses. The four selected 
variables were: a) students’ socio-economic index, b) teachers’ report on coverage of 
advanced mathematics content, c) teachers’ report on preparation to teach, and 
d) students’ perceptions of the importance of luck and innate talent in doing mathematics. 

The first step of the combined analysis explored if the selected variables had the 
expected inter-relationships. Within each jurisdiction, Pearson’s correlations were 
computed for each pair of variables; and then the pairwise rs were averaged (Table 4.2). 
Against expectations, the average correlations were low. The strongest correlation was 
between socio-economic level and importance of luck and innate talent (r = -.198). In all 
cases the direction of the correlations were according to expectations: higher socio- 
economic level was associated with increased content coverage, teacher who felt better 
prepared to teach, and students who relied less on luck and innate talent. 

Table 4.2 



Pooled Within- Jurisdictions Correlations for Four Background Variables (N = 2,219) 





Variable 


1 


2 


3 


4 


1 . 
2 . 


Socio-economic level 
Content coverage (advanced) 


. 024 









3 . 


Preparation to teach 


.077 


. 115 


— 




4 . 


Importance of luck and 
innate talent 


-.198 


-.035 


-.029 


— 



Note. Valid cases across all five jurisdictions = 88%. Analysis weighted with SENWGT. 



The low correlation among these variables was an unexpected finding. These 
correlations should be interpreted with caution. Some variables were continuous, while 
others categorical; some variables were measured at the school/class level, while others at 
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student-level. Correlations between school/class means and individual variables are 



constricted. 

As a follow up, a multivariate analysis of variance (MANOVA) was used to 
estimate to which extent the four variables combined varied as a function of the 
jurisdictions (see upper-half of Table 4.3). The omnibus test showed that there were 
significant differences among the jurisdictions in the combined set of variables, 
F( 1 6, 6755) = 96, p < .0005. The univariate analyses revealed that there were significant 
differences in all four variables depending on the jurisdiction (p < .0005). These 
associations were not very strong, with the proportion of variance accounted for the 
jurisdictions ranging from if = .26 (teachers’ preparation) to if = .08 (students’ beliefs 
in luck and innate talent). 

A multivariate test was performed comparing Chilean students with the students 
of the other four jurisdictions. As shown in the bottom-half of Table 4.3, the results 
indicated that the combined variables differed significantly between Chile and the four 
jurisdictions taken together, F{ 4, 2214) = 24, p < .0005. Univariate statistics showed that 
there were significant differences between Chilean students and students across the 
comparison jurisdictions for all variables under analysis (p < .004). The strength of the 
associations were not strong, with the proportion of variance accounted for by the 
grouping ranging from rf= .18 (teachers’ preparation) to if- .03 (students’ beliefs in 
luck and innate talent). 
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Table 4.3 

Multivariate and Univariate Results 





Wilk's df 


MS 


F 


P 


Omnibus analysis 


Multivariate test 


.53 16/6755 




96.19 


<.0005 


Univariate tests 










Socio-economic level 


4/2214 


64.94 


76.26 


<.0005 


Content coverage 
(advanced) 


4/2214 


14.21 


79 . 01 


<.0005 


Preparation to teach 


4/2214 


24.68 


204.27 


<.0005 


Importance of luck and 
innate talent 


4/2214 


17.14 


49 . 91 


<.0005 


Planned comparison: Chile 


versus the four jurisdictions 


combined 


Multivariate test 


.96 4/2214 




23 .87 


<.0005 


Univariate tests 










Socio-economic level 


1/2217 


29.93 


31.37 


<.0005 


Content coverage 
(advanced) 


1/2217 


2.00 


9 . 81 


. 002 


Preparation to teach 


1/2217 


11.35 


70 . 90 


<.0005 


Importance of luck and 
innate talent 


1/2217 


3.32 


8 . 91 


. 003 



Note. Valid cases across all five jurisdictions = 88%. Analysis weighted with SENWGT. 



Modeling Mathematics Achievement in Chile 

In this section, the aim is to understand the relationship between the background 
variables and mathematics achievement in Chile. Mathematics achievement was modeled 
at two levels: the school/class level and the students-within-schools/classes level. These 
levels matched the cluster design of the TIMSS sampling. Before modeling achievement, 
however, the distribution of the achievement scores within and among the schools/classes 
was examined. 
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Where Were the Differences in Mathematics Achievement at the 8 th Grade? 

In TIMSS 1999, Chile had a mean mathematics score of M = 395 and a standard 
deviation of SD = 84. 20 This implied that 95% of the students got between 230 and 560 
score points on the mathematics test. The partition of the achievement variance revealed 
that 39% of the differences in mathematics scores were among the schools/classes. 
Schools/classes in Chile seemed to differ more than in the comparison jurisdictions. For 
TIMSS 1995, Beaton and O’Dwyer (2002) reported that 27% of the achievement spread 
was among the schools/classes in the Slovak Republic, and just 6% in South Korea. 
While Malaysia and Miami-Dade County did not participate in TIMSS 1995, the authors 
reported that 51% of the achievement spread was among the schools/classes in the United 
States. 



In Chile, the educational system is highly segregated by social class. Results 
indicate that students from more affluent communities were clustered together in higher 
performing schools and students from socially disadvantaged backgrounds were clustered 
in lower performing schools. In fact, 43% of the socio-economic variance was among 
schools/classes. Socio-economic level was the strongest predictor of mathematics 
achievement. This variable alone accounted for 69% of achievement dispersion among 
the schools/classes. 



20 These numbers do not exactly replicate the ones in the TIMSS 1999 international report because of 
differences in the number of valid cases. 
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Among the schools/classes, the differences in average mathematics scores were 
substantial: the lowest performing school/class had an M = 298 scale points and the 
highest performing had an M = 613 scale points. The differences were so large that the 
achievement distributions of these two schools/classes did not overlap. The maximum 
score in the bottom performing school was Max = 402 scale points, the minimum score in 
the top performing one was Min = 512 scale points. 

What Factors Accounted for the Differences in 

Mean Mathematics Achievement Among the Schools/Classes? 

The research questions of this dissertation study asked about the relationship 
between background variables important for learning and mathematics achievement in 
Chile. This section addresses these questions by modeling mathematics achievement at 
the school/class level. Six hierarchical models were tested (Table 4.4). These were 
primarily cumulative models that added successive sets of explanatory variables. Each 
new set of variables focused on a different dimension of the school context: the 
implemented mathematics curriculum, teacher quality, school resources, schools’ 
community contexts, and school administration. 

The following paragraphs present the rationale underlying each model, with a 
justification of the variables used to predict achievement. The hypotheses that guided the 
analyses are stated, and the statistical results are presented and interpreted. 
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Table 4.4 



Models of Mathematics Achievement at the School/Class Level (Nj = 182) 



Model 1 


Model 2 


Model 3 


Model 4 


Model 5 


Model 6 


Among schools/classes variance ^qs. 

accounted for by model ( R 2 ) 




30% 




34% 




72% 




72% 


69% 


Reliability .95 




.95 




.94 




.87 




.87 


.88 


B (SE) 


B 


(SE) 


B 


(SE) 


B 


(SE) 


B 


(SE) 


B (SE) 


Mean mathematics score 388 (8.8) 


363 


(6.7) 


367 


(6.6) 


382 


(5.3) 


385 


(3.2) 


391 (2.7) 


IMPLEMENTED MATHEMATICS CURRICULUM 


School curriculum r „ c . 

67.8** (11.5) 

(0 = no, 1 = yes) 


67.5** 


(11.7) 


61.9* 


*(11.3) 


23.1** 


(7.9) 


22.1* 


(8.8) 




School materials/activities , _ 

. . 19 . 5** ( / . 3 ) 

(0 = no, 1 = yes) 


18.8* 


(7.4) 


17.2* 


(7.1) 


2.5 


(5.0) 








Content coverage (general) 2 5.1 (4.0) 


5.2 


(4.0) 


5.3 


(3.9) 


4.5* 


(2.2) 


4.8* 


(2.3) 




Subject emphasis 

(0 = basic, 1 = advanced) 


13.5 


(8.9) 


10.6 


(8.8) 


3.5 


(5.9) 








Additional math courses 2 -0.3 (3.8) 


-0.3 


(3.8) 


-1.6 


(3.7) 


1.5 


(2.4) 








TEACHER QUALITY 


Teacher qualification 2 


1.9 


(4.0) 


2.0 


(3.8) 


-0.6 


(2.3) 








Preparation to teach 2 


2.5 


(3.6) 


1.0 


(3.6) 


-5.1 


(2.7) 








SCHOOL RESOURCES 


Limitations in school resources 2 






-11.9* 


*(3.7) 


-0.5 


(2.4) 








SCHOOLS' COMMUNITY CONTEXT 


Socio-economic level 2 










39.5** 


(3.5) 


38.2** 


(3.1) 


44 . 1** ( 2 . 8 ) 


Urban level (0 = rural, 1 = urban) 










4 . 1 


(5.1) 








SCHOOL ADMINISTRATION 


School administration (0 = public, 1 = private) 














6.7 


(5.9) 





Note. Hierarchical linear models with random intercepts weighted with HOUWGT. 
Method: Restricted maximum likelihood with robust standard errors. 

Valid cases = 98% of the schools/classes sample. **p < .01 . *p < .05. 
z Standardized variable. 
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Model 1: The implemented mathematics curriculum. Curriculum-related variables 
can be considered the foundation for the delivery of quality instruction. For example, 
curriculum documents are often the primary method of conveying the material to be 
taught to teachers. As described in Chapter 3, the Chilean mathematics curriculum, 
distributed by the Ministry of Education, provided a framework for the schools to 
develop their own curriculum (i.e., program of study) instead of a detailed set of learning 
goals. Therefore, the official curriculum did not specify the mathematics objectives in a 
useful way for schools and classroom teachers. Moreover, there were no official 
documents providing instructional materials and activities to help the schools and 
teachers reach these objectives. Since schools had the responsibility of developing their 
own detailed objectives and making decisions about their instructional materials, great 
variation was expected in the way the schools implemented the official curriculum. It was 
expected that a greater coverage of TIMSS mathematics content topics in the 
implemented curriculum would be associated with higher mathematics achievement. Due 
to these assumptions, priority was given to the inclusion of variables measuring the 
implemented curriculum in the first model tested in this study. 

Model 1 analyzed the relationship between mathematics achievement and five 
curriculum-related variables: school curriculum (dummy variable), school instructional 
materials/activities (dummy variable), coverage of mathematics content, subject emphasis 
(dummy variable), and provision of additional mathematics courses. 
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This study hypothesized that schools that had their own curriculum and 
instructional materials/activities would attain higher mathematics scores than schools that 
did not. The content coverage variable measured the “depth and breadth” with which the 
teachers covered the mathematics content topics assessed by TIMSS 1999. It was 
hypothesized that increased content coverage would lead to increased mathematics 
performance. In teaching mathematics, some teachers emphasized “mainly numbers,” 
while others emphasized more advanced areas like algebra and geometry. This study 
hypothesized that increased emphasis on advanced content areas would be associated 
with higher mathematics achievement. There is a wide range of mathematics knowledge 
and skills among students in the same schools/classes. In this dissertation study it was 
hypothesized that schools providing additional mathematics courses (i.e., remedial, 
enrichment) to address the needs of students with different performance levels would 
attain higher mathematics scores than schools that did not. 

In Model 1, the characteristics of the implemented mathematics curriculum 
accounted for 30% of the achievement variance among the schools/classes. Of the five 
indicators of curriculum implementation, two were highly significant. These were the 
schools/classes that had their own mathematics curriculum (p < .0005), and the 
schools/classes that had their own instructional materials/activities (p < .009). These 
results confirm the hypothesis that teachers in schools that had developed their own 
mathematics curriculum, as well as instructional materials and activities, would be in a 
better position to implement effective mathematics instruction. 
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It appears that having a detailed curriculum and adequate instructional materials 
go a long way toward ensuring content coverage. Content coverage and subject emphasis 
both were significant predictors when entered alone in the equation (p < .049; not shown 
in Table 4.4), but were not significant predictors of achievement when combined with 
schools having their own curriculum and instructional materials. Thus, their lack of 
significance in Model 1 must be interpreted as the consequence of their inter-correlation 
with school curriculum and instructional materials. In schools having their own 
curriculum and instructional materials, teachers were likely to cover more mathematics 
content topics and to emphasize more advanced topics. 

Contrary to expectations, the provision of additional mathematics courses was not 
a significant predictor of mathematics achievement. This was in fact the case in all 
models tested in this dissertation study. More information about the nature of these 
courses is needed to interpret this finding. 



Model 2: Teacher quality. Since teachers have the responsibility of delivering the 
mathematics curriculum at the classroom level, they play a key role in mediating between 
the curricular policies and the students’ learning processes. Because research has shown 
that teacher quality and achievement are positively related (Mayer et al., 2000; Mullis, 
Martin, Gonzalez, et al., 2000), there is concern in Chile about how well prepared 
teachers are to fulfill their role. 
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In Model 2, indicators of teacher quality were combined with the implemented 
curriculum variables. The new indicators were: 

1. Teacher qualification. This variable measured a) how far teachers advanced in 
their formal education; b) if they had a teacher certificate; and c) if they 
majored in mathematics in their undergraduate and/or graduate programs. 

2. Teacher preparation. This variable measured how well prepared the teachers 
felt to teach different mathematics content. 

In this study it was hypothesized that the more qualified teachers and the teachers 
who felt better prepared to teach a variety of mathematics content topics would have 
students with higher mathematics achievement. 

The percentage of variance accounted for by Model 2 remained the same as the 
previous model (30%). In combination with the curriculum-related variables, neither of 
the teacher quality variables made a significant contribution to explaining variation in 
achievement. School curriculum and school materials/activities remained the only 
significant predictors of achievement (p < .012). 

Nevertheless, it should be emphasized that teacher preparation was a significant 
predictor of achievement when analyzed alone (p = .044, not shown in Table 4.4). 
However, when school curriculum, school materials/activities, and content coverage were 
included with teacher preparation, the contribution of this variable was negligible. It 
appears that better prepared teachers were more likely to work in schools that had their 
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own math curriculum and materials/activities; these teachers were also more likely to 
cover more mathematics content in classes. 

In this dissertation study, it was expected that higher-qualified teachers would 
have higher-achieving students in mathematics. However, a closer look at the data 
showed that all teachers in Chile had basically the same qualification level - a bachelor 
university degree with a teacher-training certificate. The lack of variance in teacher 
qualification level probably explains its lack of significance in explaining between-school 
variation in mathematics achievement. 



Model 3: School resources. The provision of quality mathematics instruction can 
be facilitated by availability of school resources, such as instructional space, libraries and 
computers. Shortages or inadequacies of these resources may affect the school capacity to 
provide quality instruction (Mullis, Martin, Gonzalez, et al., 2000). In Chile, where 
schools are organized into different systems, it was expected that this variable could have 
a substantial impact on achievement. School resources are unevenly distributed between 
the public and private systems, with the private systems having more resources. 

Because some schools have more resources than others, the third model 
encompassed the curriculum and the teacher variables, plus a variable measuring the 
extent to which limitations in resources affected the schools’ capacity to provide 
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instruction. It was hypothesized that limitations in school resources would have a 
negative partial effect on the achievement outcome. 

Model 3 accounted for 34% of the achievement variance among the 
schools/classes, i.e., four percentage points more than the previous models. According to 
the given hypothesis, limitations of school resources had a negative effect on 
achievement, having controlled statistically for the effect of the other predictors in the 
model ( b = -11.9, p = .002). School curriculum and school materials/activities remained 
significant (p < .016), while the other predictors in the equation did not. 



Model 4: Schools’ community contexts. All previous models looked at 
achievement in relation to school-related variables, without taking into account the effect 
of socio-economic level. However, it is well known that mathematics achievement is 
strongly related to the socio-economic level of the communities served by the schools 
(Beaton & O’Dwyer, 2002; Coleman et al., 1966; Ramirez, 2003). The importance of 
socio-economic level in Chile could be even greater considering that the school systems 
are strongly segregated by social class (Mella, 2003). 

Beyond the socio-economic level, the urban/rural status of schools may also affect 
their capacity to provide quality instruction. Rural schools usually work in more socially 
deprived contexts and are more isolated than urban schools. In Chile, rural schools have 
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lower academic achievement than urban schools serving communities with the same 
socio-economic background (Mizala & Romaguera, 2000; Ramirez, 2003). 

Because of the well-documented importance of socio-economic level. Model 4 
tested the simultaneous effect of two school community variables - socio-economic level 
and urban/rural status of the school - together with the previously described indicators of 
the implemented curriculum, teacher quality, and limitations of school resources. This 
model accounted for 72% of the mean achievement differences among the 
schools/classes, which was more than double the amount accounted for in the previous 
model. The partial regression coefficient of socio-economic index was positive and 
highly significant ( b = 39.5, p < .0005). This result vividly illustrates the consequences of 
having Chile’s school systems so closely aligned with socio-economic status. 

The urban/rural status of the school community was not a significant predictor of 
achievement, when analyzed together with socio-economic level and other predictors in 
the equation (p = .42). This was most likely because more affluent schools tend to be 
located in urban areas and poorer schools in rural areas. 

Schools having their own mathematics curriculum remained a significant 
predictor in Model 4 (p = .004). However, in combination with stronger socio-economic 
indicators, schools having their own instructional materials/activities and adequate 
resources had less explanatory power. Instead, greater content coverage emerged as 
having a significant effect on mathematics achievement (p = .04). With the structure of 
the Chilean system so closely aligned with socio-economic status, it could be easy to 



129 




overlook the importance of teaching an effective curriculum. Yet, in each subsequent 



analysis, the importance of this factor is apparent. 



Model 5: School administration. Model 5 evaluated the combined effect of the 
four most powerful predictors: school administration (public/private), school socio- 
economic level, schools having their own curriculum, and extent of content coverage. 
These variables accounted for 72% of the variance between schools/classes in 
mathematics achievement - the same proportion than the previous model. 

As explained previously, schools in Chile are organized and administered by 
different systems that are directly associated with socio-economic background. The 
public schools tend to serve the students from the lower end of the socio-economic 
distribution, while private schools (private-subsidized and elite-paid) serve the students 
from the middle and upper end of the distribution. Thus, it was important to examine the 
effect of school administration (public/private) in combination with socio-economic level 
and curriculum variables to see if type of school system, in and of itself, explained any 
variation in mathematics achievement. 

It also was interesting to examine the impact of curriculum and content coverage 
in relation to the type of school administration and socio-economic level. It would seem 
that the schools in the private system would be more likely to be able to afford to produce 
their own curriculum and facilitate greater classroom coverage of mathematics content, 



130 




while the schools in the public system may have had to struggle with the less specific 
national curriculum. 

Among the variables included in Model 5, as anticipated, socio-economic level 
was the strongest predictor of mathematics achievement ( b = 38.2, p < .0005). However, 
school curriculum and content coverage also had significant effects on mathematics 
achievement ip = .012 and p = .034, respectively). The model predicted that a school with 
its own curriculum would score 22 score points higher than a similar school without its 
own curriculum, holding constant the other variables in the model. This was a substantial 
difference considering that the achievement gap between 7 th and 8 th graders in Chile was 
33 score points (Ramirez, 2004). 

Finally, an additional model was built to examine how much added variance was 
accounted for by the fifth model, in comparison to a model that only used socio- 
economic level as a predictor. The socio-economic level of the school community 
accounted for 69% of the between-school/class variance in mathematics achievement 
(Model 6 in Table 4.4), almost the same percentage (72%) that was accounted for by the 
model that evaluated the combined effect of school administration, school curriculum, 
content coverage, and socio-economic level. This finding suggests that, in Chile, school 
assets are distributed along socio-economic lines. Schools working in socially advantaged 
backgrounds are more likely to have their own curriculum and their teachers are more 
likely to have covered more mathematical content in their classes. 
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What Factors Accounted for the Differences in Mathematics Achievement 
Among Classmates? 

All the models presented in the previous section attempted to explain the 
achievement variance at the school/class level. They used predictors that were constant 
for all students within a school/class. As a consequence, these models were not designed 
to account for the achievement dispersion that was within the schools/classes. 

Results indicate that, in Chile, 61% of the achievement variance was within the 
schools/classes. Classmates covered a wide range of achievement scores. In a typical 
middle-performing school/class, mathematics scores approximately ranged from 250 to 
524 score points. On average, the schools/classes had an achievement range of 274 score 
points. This average was equivalent to slightly more than three standard deviations in the 
national achievement distribution. This implies that students who could apply their 
knowledge and skills to solve mathematical problems and students who could not even 
do basic computations with whole numbers, shared the same schools/classes. 

This section aims to further understand the achievement differences among 
classmates in the same schools/classes. Since students’ attitudes toward mathematics are 
expected to vary among classmates, they were used as predictors of mathematics 
achievement at the students-within-schools/classes level. In this dissertation study it was 
hypothesized that students with more positive attitudes toward mathematics had higher 
mathematics scores. 
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Chilean students reported very positive attitudes toward mathematics. Practically 
all (99%) endorsed the statement ( agreed or strongly agreed) “it is important to do well 
in mathematics,” and 73% reported liking this subject area. Considering the poor average 
performance of Chile in the TIMSS 1999 test, it was surprising that 41% of the students 
reported that “mathematics is an easy subject,” while up to 77% endorsed the statement 
“I usually do well in mathematics.” Half of the students (51%) reported that “to do well 
in mathematics you need good luck,” and 43% aspired to “finish university.” As 
mentioned in the comparative results section, students may have answered in a socially 
desirable way to some of these statements. 

As explained in Chapter 3, most of these source variables were combined into 
indicators that measured different facets of the attitude construct. Five indicators were 
used as predictors in the achievement models: a) liking mathematics, b) importance of 
mathematics, c) difficulty in doing mathematics, d) importance of luck and innate talent 
in doing mathematics, and e) educational expectations. 

Table 4.5 presents the predictive models at the students-within-schools/classes 
level. Model 1 used the five attitudes variables, and accounted for 22% of the within 
schools/classes achievement variance. The results only provided partial support for the 
hypothesis that students’ attitudes toward mathematics were positively related to 
mathematics achievement. Only three of the five predictors had a significant partial 
association with the outcome. 
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According to expectations, students reporting that mathematics were more 
difficult for them had significantly lower mathematics scores than their classmates, 
conditional on the other variables in the model ( b = -21.7, p < .0005). This finding is 
consistent with those reported by Howie (2002) in South Africa and by Nasser and 
Birenbaum (2004) in Israel. 



Table 4.5 

Models of Mathematics Achievement at the Students-Within-Schools/Classes Level 
(N = 5,702) 



Model 1 



Model 2 



Rx 



R 



22.3% 



. 96 



22 . 4 % 



. 96 



Variables 



(SE) 



( SE ) 



Mean mathematics score (intercept) 


391 (4.2) 


391 (4.2) 


Liking mathematics 


-0.9 (2.9) 


-0.9 (2.9) 


Importance of mathematics 


-0.1 (1.3) 


-0.2 (1.4) 


Difficulty in doing mathematics 


-21 . 7** (1.7) 


-21 .6** (1.7) 


Importance of luck and 
innate talent 


-11 . 1** (1.9) 


-11.0** (1.9) 


Educational expectations 


10.1** (1.2) 


9 . 9** (1.3) 


Socio-economic level 




1.3 (3.1) 


Note. All the predictors were standardized and group-mean centered. 


Method: Restricted 



maximum likelihood with robust standard errors. Analysis weighted with HOUWGT. 
Valid cases = 97% of the student sample. **p < .0005. 



Also according to expectations, students reporting a stronger belief in luck and 
innate talent also attained significantly lower scores than their classmates, net the effect 
of the other variables in the model (b = -11.1, p < .0005). This result is consistent with 
the literature on locus of control and with previous studies using the TIMSS data 
(Benham, 1995; Stemler, 2001). 
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There was a positive partial association between students’ educational 
expectations and the outcome, hence students who expected to go further in their formal 
education (e.g., finish university) attained higher mathematics scores than their 
classmates with lower expectations ( b = 10.1, p < .0005). Again, this finding was 
consistent with previous studies of educational attainment (Martin, Mullis, Gregory, et 
al„ 2000). 

In Chile, liking mathematics and the importance of mathematics were not 
significant predictors of mathematics achievement, once the other variables in the model 
were held constant (p > .77). This was an unexpected finding that did not support the 
results of previous studies (Howie, 2002). The lack of significance of these indices may 
be better understood bearing in mind the tendency of Chilean students to report very 
positive feelings toward mathematics, resulting in the low variability of these predictors. 

Model 2 was used to test the partial effect of the students’ socio-economic level 
on achievement, when analyzed together with the attitudes variables. The same predictors 
reported as significant in the first model were also significant in Model 2. These were 
difficulty of doing mathematics, importance of luck and innate talent, and educational 
expectations. Results also indicate that classmates with a more advantaged socio- 
economic background did not attain higher mathematics scores than their classmates with 
more disadvantaged backgrounds, after controlling statistically for the other variables in 
the model (p = .68). This finding is consistent with previous studies that show that once 
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students are allocated to classes and schools, their cultural background is not a significant 
predictor of academic performance (Beaton & O’ Dwyer, 2002). 

Summary of Findings 

This dissertation study found that Chilean students had a more socially 
disadvantaged background than the students in South Korea, Malaysia, the Slovak 
Republic, and Miami-Dade County Public Schools. Chilean schools were also more 
economically segregated than the schools in the comparison countries. Chilean students 
were taught significantly fewer advanced mathematics topics than the students across the 
four comparison jurisdictions. While most of the Chilean students were taught by 
teachers who emphasized “mainly numbers,” less than one-third of the students in the 
other jurisdictions were taught by teachers doing so. Chilean students were also taught by 
teachers who felt less prepared to teach a variety of mathematics topics, when compared 
to the students across the four jurisdictions. These findings confirm the hypotheses of this 
study. 



However, other results ran against the stated hypotheses. Chile did not stand apart 
from the other jurisdictions in schools’ having their own curriculum, schools’ having 
their own instructional resources, schools’ offering additional mathematics courses, 
school resources, and teacher qualifications. Chilean students did not have more negative 
attitudes toward mathematics, when compared to the students across the other 
jurisdictions. 
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To study impact of important education variables in relation to mathematics 
achievement in Chile, several predictive models were tested using hierarchical linear 
models. At the school/class level, a model that included school-related indicators 
(implemented mathematics curriculum, teacher quality, and school resources) and 
schools’ community variables (socio-economic level and urban/rural location) accounted 
for 72% of the achievement variance. Another model that only included a socio- 
economic variable accounted for 69% of the achievement variance among the schools. 
This finding suggests that, in Chile, school assets are unevenly distributed. Schools 
serving more affluent communities were more likely to have their own curriculum, better 
prepared teachers, and better instructional resources, when compared to schools working 
in more socially deprived contexts. 

Because school and community factors were closely interwoven, their unique 
effect on the achievement outcome could not be isolated. Nevertheless, two school- 
related variables were shown to have a significant partial effect on the outcome: school 
curriculum and content coverage. This result suggests that schools having their own 
mathematics curriculum and whose teachers covered more content attained significantly 
higher mathematics scores than schools without their own curriculum and whose teachers 
covered less content in classes. This achievement gap holds true when comparing schools 
having the same type of administration (public/private) and working in similar social 
contexts. 
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At the students-within-school/class level, students’ expectations for further 
education, perceived difficulty of doing mathematics, and beliefs in luck and innate talent 
were significant predictors of mathematics performance. Liking and the importance of 
mathematics were not associated with achievement. This was mainly the consequence of 
almost all the Chilean students reporting that they liked mathematics and found this 
subject important. 
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CHAPTER 5 



CONCLUSIONS 

This chapter begins with an overview of the problem under investigation, 
followed by a statement of the main purposes of this dissertation study. Next, central 
findings are presented. Results are discussed and policy implications for the Chilean 
education system are raised. 

The TIMSS 1999 test revealed that the mathematics knowledge and skills of 
Chilean 8 th graders were far below than expected. Only half of the students showed 
evidence of mastering basic computations with whole numbers, and just 15% could apply 
mathematical concepts in straightforward situations. The highest performing students 
scored similarly to the average students in South Korea, Malaysia, and the Slovak 
Republic (Mullis, Martin, Gonzalez, et al., 2000, chap. 1). Chilean students lagged a 
whole school year behind when compared to the students in Miami-Dade County Public 
Schools (see Ramirez, 2004). 

This dissertation study aimed to understand better why Chile was lagging behind 
in educational achievement when compared to other countries with similar economic 
conditions, or when compared to United States districts serving high proportions of low- 
income students. Mathematics achievement and background characteristics of the Chilean 
students were compared to similar data from South Korea, Malaysia, the Slovak 
Republic, and Miami-Dade County. This dissertation study also aimed to better 



139 




understand how important characteristics of the Chilean educational system could 



account for poor student achievement in mathematics. 



The results of this study suggest that commonly used economic indicators 
(e.g., gross national product) veil important differences in the cultural capital of the 
nations. Chilean students come from a more disadvantaged social background than the 
students in South Korea, Malaysia, the Slovak Republic, and Miami-Dade County. 
Chilean students have fewer educational resources at home (e.g., books, computer, study 
desk), and their parents have less education than those in the comparison jurisdictions. 
For instance, while slightly more than one-third of the Chilean students reported that their 
mothers finished secondary school, this proportion reached 50% in Malaysia. In South 
Korea, the Slovak Republic and Miami-Dade County, parents have even more years of 
education than parents in Malaysia. The low cultural capital of the Chilean families 
seems to be accounting for an important proportion of the achievement differences 
between Chile and the comparison jurisdictions. 

Chile only expanded its educational system recently. As a consequence, Chilean 
families have a poor cultural capital. Chile reached almost universal enrollment in 
primary education during the 1990s. In South Korea, the Slovak Republic, and the United 
States, this goal was accomplished during the 1950-60s. It seems that Chile needs to 
overcome the educational deficits of the previous generation. 
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Because of the poor educational level of the Chilean population students’ parents 
are not in a position to provide strong educational support at home. Nevertheless, parental 
support at home is important for the students to learn at school (Reynolds & Teddlie, 
2000). Chile needs to find viable ways to maximize parental involvement. One way to do 
that may be to provide parents with more and better information about their child’s 
school. For instance, parents may receive school reports with national and local norms, 
and with the percentage of students reaching proficiency levels. 

Parental involvement may also be increased by providing parents with reports of 
how their children are doing in the school according to the curricular objectives. The 
pervasive grade inflation existing in Chile is probably playing against parental 
involvement. As noted by Stevenson and Stigler (1992), when parents are (wrongly) told 
that their child is doing well in the school, this creates a sense of satisfaction that 
becomes an obstacle for improving education. Setting clear performance standards and 
more rigorous evaluation criteria in the classrooms may indirectly contribute to engage 
Chilean parents in their children education. 

The socio-economic level of the students’ families (which takes into account 
parents’ education) is strongly related to mathematics performance in Chile. Schools 
serving socially advantaged communities show substantially higher mathematics scores 
than schools working in socially deprived areas. This achievement gap is, in part, the 
outcome of the differential educational resources students have at home. A student would 
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be in a better position to learn at school if he/she has literate parents who can help with 
the schoolwork, and if he/she has books and a computer at home. 

However, the achievement gap among Chilean schools is also impacted by the 
unequal distribution of school assets between high- and low-income schools. This study 
found that schools in more affluent communities are more likely to have more and better 
instructional resources and also more likely to hire better prepared teachers. These 
teachers, in turn, are more likely to emphasize advanced mathematics topics. Schools 
serving affluent communities are also more likely to develop their own curriculum or 
program of study, instructional materials, and activities; these are important resources 
since the national curriculum provided little guidance on the subject of what to teach and 
how to teach it. These findings are consistent with previous studies reporting the unequal 
distribution of school resources (and opportunities to leam) among schools, and the 
correlation between school resources and socio-economic level. As stated by Gau (1997), 
low-income students’ disadvantaged situation is compounded as they receive less 
educational support at home and have fewer opportunities to learn at their local schools. 
To provide low- and middle-income students with access to quality education is a 
pending task in Chile. 

Another relevant finding of this dissertation study is that Chile has the most 
economically segregated educational system, when compared to South Korea, Malaysia, 
and the Slovak Republic. Chilean students vary widely in their socio-economic levels, 
and this variance is clearly structured in the school system. In general terms, public 
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schools serve the lower-class, private-subsidized schools serve the middle-class, and the 
elite -paid schools serve the upper-class. While economic segregation in the school system 
is a common phenomenon around the world (Beaton & O’Dwyer, 2002), none of the 
comparison countries have the degree of segregation observed in Chile. 

South Korea offers an interesting example of a highly privatized school system 
that has no economic segregation. In this country, one-third of the schools are private but 
funded by the state (similar to the private-subsidized schools in Chile). Both public and 
private schools demonstrate very advanced (and very similar) academic performance. 
The motto behind this success seems to be that the same rules of the game apply for all 
the schools that receive public funds, no matter if public or private. For instance, Korean 
students are assigned at random to a school in their residential area, regardless if it is 
public or private. Korean parents have to pay a fee from grade 7 onward in all state 
funded schools. The only difference between public and private schools is that the former 
are administered by the local government and the latter are administered by religious 
congregations. 



In this study, special attention was given to the mathematics curriculum and the 
role it played in explaining the poor performance of Chilean students. At the time of the 
TIMSS 1999 data collection, the official curriculum in Chile had no more than three 
pages. This document listed the mathematics objectives to be reached by Chilean 8 th 
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graders. The curriculum was deficient; it was the least specific when compared to the 
curriculum of the other jurisdictions, and covered the least content and cognitive skills. 

The idea behind this limited curriculum was that the schools would use it by 
adding their own curricular proposals or programs of study. The Chilean curriculum was 
more of a framework for schools to develop their own curriculum, than a curriculum in 
itself. In practice, only a small group of schools developed their own curriculum (around 
16%); most of these schools were from the elite-paid and private-subsidized systems. 
Public schools, which served a more socially deprived population, lacked the necessary 
resources to develop their own curriculum. 

It seems that having a school curriculum was necessary to operationalize the 
curricular mandates. Not surprisingly, schools having their own curriculum performed 
significantly better than the schools that did not. The achievement gap between the 
former and the latter was equivalent to two-thirds of a school year. This difference held 
true after controlling statistically for the schools’ socio-economic level and type of 
administration (public/private). 

Nowadays, Chilean schools have a more rigorous and detailed national 
curriculum than in 1998, when data was collected for this dissertation study. The new 
curriculum may serve as a leading force to improve other important areas of the 
educational system. By providing clearer performance standards, the new curriculum 
should better support teachers in choosing what to teach, in preparing their classes, and in 
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grading their students’ performance. The Ministry of Education should closely monitor 
the extent in which the new curriculum is changing teaching practices. 

The new curriculum could be used to boost teacher-training programs. 
Universities should prepare future teachers in how to teach the reformed curriculum. A 
teacher certification exam may be used to ensure that the future teachers master the new 
curriculum, and that they are prepared to teach the advanced mathematics knowledge and 
skills stated in it. 



Beyond the official curriculum, it is the curriculum delivered in classes that 
actually shapes students’ opportunities to learn mathematics. In Chile, the gap between 
the official curriculum and the “delivered” one is substantial. While 8 th grade students 
should be studying geometry and algebra, most of them study basic mathematics content 
in their classes (Mullis et al., 2000, chap. 5). An important finding of this dissertation 
study is that Chilean students have significantly fewer opportunities to learn advanced 
mathematics content, when compared to the students across South Korea, Malaysia, the 
Slovak Republic, and Miami-Dade County Public Schools. 

The lack of rigor and detail of the pre-reform curriculum is partly to blame for 
this poor content coverage. However, it is possible that Chilean teachers do not introduce 
new content because of the already poor mathematics skills of their students. While 
slowing down to match the skills of students may be appropriate, it is necessary to 
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highlight the tradeoffs of this approach. It seems that Chilean students are not provided 
with enough opportunities to learn the advanced mathematics content stated in the 
national curriculum. The advanced mathematic content is what standardized tests 
regularly assess. 

Another factor that may be contributing to the poor coverage of mathematics 
content is that Chilean teachers do not feel prepared to teach it. If teachers teach what 
they barely know, the quality of instruction will suffer. Chilean students are taught by 
teachers who report feeling somewhat prepared to teach a variety of mathematics content 
- including algebraic representations, coordinate geometry, and data analysis. In South 
Korea, Malaysia, the Slovak Republic, and Miami-Dade County Public Schools, students 
are taught by teachers who feel significantly better prepared to teach mathematics. This 
situation urges the Chilean authorities to pay more attention to pre- and in-service teacher 
preparation programs. 

Chilean teachers report substantial variation in the amount of mathematics content 
covered in their classes. This dissertation study found that there is a positive relationship 
between content coverage and mathematics performance. This relationship holds true 
after controlling statistically for schools’ socio-economic level and type of administration 
(public/private). This finding is consistent with previous studies reporting a positive 
relationship between students’ opportunities to learn and mathematics achievement (Gau, 
1997; Secada, 1992). 
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Because the analyses presented in this cross-sectional study are correlational in 
nature, the association between variables should not be interpreted as definite causal 
effects. Correlations may suggest a causal relationship but, ultimately, further research is 
required to establish cause and effect. Hence, causal interpretations should be interpreted 
with prudence. 

Self-report surveys like TIMSS are useful to study the opinions and beliefs of 
students, their teachers and principals. However, these instruments also have limitations 
that the reader needs to be aware of. Some students may answer in a socially desirable 
way to some questions, or may have a tendency to agree with every single statement in a 
questionnaire. It seems that these tendencies are somewhat stronger in countries with low 
academic achievement like Chile. Another limitation is that respondents from different 
cultures tend to answer in somewhat different ways to the same questions. For instance, 
Korean students seemed to be very strict to evaluate their mathematics skills, whereas 
Chilean respondents seemed to be more lenient in evaluating themselves. Similarly, 
Chilean principals were very generous in evaluating their school resources; this was 
probably because their schools have recently benefited from increased resources. In 
contrast, Korean principals seemed to have higher expectations about what the school 
resources should be. 



In Chile, the debate about how to improve academic achievement usually points 
to structural factors such as school resources, the curriculum, or teacher quality. Little 
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attention has been given to students’ characteristics such as how much they like 
mathematics, how do they perceive themselves doing mathematics (self-perception), and 
what factors they believe affect their mathematics performance. 

The findings of this study show that Chilean students’ attitudes toward 
mathematics are very positive. Practically all of them think it is important to do well in 
this area, and three-fourths report liking mathematics. These are good strengths for 
learning and instruction. Unfortunately, Chilean students seem to have an extremely high 
and distorted self-perception of their knowledge and skills. Despite their low academic 
performance, 77% report that “I usually do well in mathematics,” and 41% think that 
“mathematics is an easy subject.” 

Mathematics difficulty and self-perception in doing mathematics are probably a 
function of the difficulty level of the contents covered in classes. Students might perceive 
mathematics as an easy subject because they see variations of the same (basic) content 
during the school year. Another plausible explanation is that students think that they do 
well in mathematics simply because they get good grades in this subject. Grade inflation 
and poor academic expectations are a pervasive problem in Chilean schools (Arancibia, 
1994; Riveros, 2004). 

Chilean students have stronger beliefs in the importance of luck and innate 
intelligence, when compared to students across South Korea, Malaysia, the Slovak 
Republic, and Miami-Dade County Public Schools. A relevant finding was that, in Chile, 
students who rely more on luck and innate intelligence attained significantly lower 
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mathematics scores than their classmates who gave less importance to these factors. This 
result is consistent with the locus of control theory, which states that the more the 
students rely on factors out of their control (e.g., luck, innate talent), as opposed to more 
controllable factors (e.g., effort, motivation), the lower their academic performance 
(Benham, 1995). In contrast, Korean and Malaysian students are embedded in Confucian 
ideas of effort and perseverance and therefore are less likely to emphasize the importance 
of good luck and innate intelligence in doing well in mathematics. 

In Chile, only the elite class has access to higher education. Nevertheless, an 
impressive 43% of the students aspire to a university degree. Chilean students who plan 
to finish university attain significantly higher mathematics scores than students who do 
not plan to do so. This finding is consistent with previous studies of academic 
achievement (Martin, Mullis, Gregory, et al., 2000). 

Educational systems are extremely complex social organizations. They respond to 
social goals, and are shaped by historical and present circumstances. This dissertation 
study does not pretend to encompass all this complexity. Many important dimensions of 
school systems were not considered in the analyses presented here (e.g., classroom 
practices, school climate). This dissertation study focused on a subset of variables 
relevant for the analysis of mathematics performance. It is hoped that the findings 
reported will be useful in understanding the poor academic performance of Chilean 
students. It is also hoped that these findings will be useful to inform the debate in Chile 
about how to improve mathematics education. 
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